&,T>  siiq-ev-oi 


OF  "QPV 


Automatic  Line  Network  Extraction 
^  from  Aerial  Imagery  of  Urban  Areas 
through  Knowledgefcased  Image  Analysis 


CM 

CM 

CM 

< 

I 

D 

< 


Final  Technical  Report 


December  1 989 


United  States  Army 

Research,  Development  and  Standardization  Group  UK 
London  England 


DTIC 


Contract  Number:  DAJA  45-86-C-0049 


FGAN-FIM,  Eisenstockstr.  12 
D-7505  Ettlingen  6,  W.  Germany 

•OriMtVtX  MtfMSM  - 

AU  mC 

lova  *AU  *•  *»  *»•**  "** 

Approved  for  Public  Release;  distribution  unlimited 


90  06  11 


Automatic  Line  Network  Extraction 
from  Aerial  Imagery  of  Urban  Areas 
through  Knowledge  Based  Image  Analysis 


Final  Technical  Report 

H.  Fiiger,  K.  Jurkiewicz,  H.  Kazmierczak, 
B.Nicolin,  W.Ott 


December  1989 

United  States  Army 

Research,  Development  and  Standardization  Group  UK 
London  England 

Contract  Number:  DAJA  45-86-C-0049 

Contractor:  Forschungsgesellschaft  fur  Angewandte  Naturwissenschaften 

(FGAN) 

Wachtberg,  W.  Germany 

Principal  Investigator:  Prof.  Dr.- Ing.H.  Kazmierczak 
Forschungsinstitutfur  Informationsverarbeitung  und  Mustererkennung  (FIM) 
Eisenstockstr.  12,  D-7505  Ettlingen,  W. Germany 

FIM  Report  No.  209 


Accesion  For 


it 

□ 

□ 


NTIS  CRAS.I 
DTIC  TAB 

Unannounced 

Justification 


By _ 

Distribution  f 


Availability  Codes 


Dist 


M 


Avdd  und/or 
Special 


Th^Research  reported  in  this  document  has  beenjiiada^aossible  through 
the  ^oport  and  sponsorship  of  the  U.S.G0tf£rnment  throughlts'Utjited 
States  wmy  Research,  Development  apdStandardization  Group  UK. 
report  is  inteq^ed  only  for  the  interpalmanagement  use  of  the  Contractor'1 

and  the-tf?S.  Government 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMP  No.  0704  0188 

Exp  Date:  Jun  30.1966 

la. REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

1b  RESTRICTIVE  MARKINGS 

2a  SECURITY  CLASSIFICATION  AUTHORITY 

3  DISTRIBUTION/AVAILABILITY  OF  REPORT 

APPROVED  FOR  FUBLIC  DISTRIBUTION 

UNLIMITED 

2b .  DECLASSIFICATION/DOWNGRADING  SCHEDULE 

4. PERFORMING  ORGANIZATION  REPORT  NUMBER|S) 

FIM-NO.  209 

5  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 

R&D  5679-EN-Ol 

6a  NAME  OF  PERFORMING  ORGANIZATION 
Forschungsinst itut  fur  Informa- 
t ionsverarbe i tung/Mustererkennung 

6b  OFFICE  SYMBOL 
(If  applicable) 

FGAN-FIM 

7a. NAME  OF  MONITORING  ORGANIZATION 

European  Research  Office.  USARDSG-UK 

6c  ADDRESS  (City .Stale .and  Zi°  Code ) 

FGAN-FIM ,  D-7505  Ettl ingen  6. 

Eisenstockstr . 12 ,  West  Germany 

7b  ADDRESS  fCHy. Stale,  and  ZIP  Code) 

Box  65,  FPO  NY  09510-1500 

8a. NAME  OF  FUNDING/ SPONSORING 
ORGANIZATION  gUR  r&£) 

USARDSG-UK 

8b  OFFICE  SYMBOL 
(If  applicable) 
AMXSN  UK-RE 

9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 

DAJ A45 -86- C -0049 

8c. ADDRESS  (City.  State  and  ZIP  Code) 

Box  65.  FPO  NY  095101500 

10. SOURCE  OF  FUNDING  NUMBERS 

PROGRAM 
ELEMENT  NO 

6I102A 

PROJECT 

NO  1LI61I- 
02BH57 

TASK 

NO. 

01 

WORK  UNIT 
ACCESSION  NO 

11  TITLE  (Include  Security  Classification) 

(U)  Automatic  Line  Network  Extraction  from  Aerial  Imagery  of 
Urban  Areas  through  Knowledge  Based  image  analysis 


12  PERSONAL  AUTHOR(S) 

Prof.  Dr.  H.  Kazmierczak 


13a.  TYPE  OF  REPORT 

13b  TIME  COVERED 

14  DATE  OF  REPORT 

- p> - 

15  PAGE  COUNT 

Final 

FROM  Nov  86  TO  Aug  89 

Aug  89 

152 

16  SUPPLEMENTARY  NOTATION 

Image  processing,  structural  pattern  recognition,  blackboard  oriented  symbolic 
processing,  knowledge  based  image  analysis,  image  understanding,  aerial  imagery,  urban  area* 


Liz. _ 

\ 

10  SUBJECT  tfpms  (Continue  on  reverse  If  necessary  and  identify  by  block  number) 

■F  ■■ 

SmS 

19  ABSTRACT  (Continue  on  reverse  if  necessary  and  identity  by  block  number) 

\  Different  methods  for  automatic  detection  of  line  objects  applied  to  aerial  images  to  extract 
streets  from  urban  scenes  are  investigated.  First,  test  results  achieved  from  two  existing 
,  methods  of  low  level  iconic  image  processing  by  stream  following  (line  tracking)  and 
structured  parallel  operations  (image  filtering,  feature  extraction)  ore  given  Second,  a 
medium  level  iconic  image  processing  method  developed  for  edge  and  area  segmentation  is  de¬ 
scribed  and  results  from  image  segmentation  are  presented  symbolically.  Then  two  preliminary 
approaches  of  high  level  symbolic  processing  by  knowledge  based  blackboard  oriented  structure 
analysis  are  tested.  One  is  originating  with  preprocessing  by  low  level  edge  filtering,  the 
other  by  medium  level  area  segmentation.  First  results  from  the  image  understanding  method  for 
street  network  extraction  are  presented, 


20  DISTRIBUTION/ AVAILABILITY  OF  ABSTRACT 

0  UNCLASSIFIED/UNLIMITED  P  SAME  AL  RPT  PdTIC  USERS 

21  ABSTRACT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

22«.NAME  OF  RESPONSIBLE  INDIVIDUAL 

J.C.  COMAT I 

22b. TELEPHONE  (include  Area  Code ) 
01-402  7331 

22c  OFFICE  SYMBOL 
AMXSN-UK-RE 

00  FORM  1473,  61  MAP,  83  APR  edition  may  be  used  until  exhausted 
AH  other  editions  are  obsolete 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 
UNCLASSIFIED 


1 


Contents 

Report  Documentation  Page  with  Abstract  i 

Contents  ii 

List  of  Figures  iv 

Summary  vii 

1  Introduction  1 

1.1  Problems  .  1 

1.2  New  Approach .  2 

1.3  Test  Imagery .  4 

1.4  Implementations .  5 

1.5  Investigations .  9 

2  Extraction  of  Line  Objects  by  Stream  Following  11 

2.1  The  Line  Extraction  Method  . 11 

2.2  Line  Extraction  from  Urban  Scenes . 12 

2.2.1  Test  Imagery . 12 

2.2.2  Parameter  Adjustment .  14 

2.2.3  Results  of  Line  Extraction . 14 

3  Extraction  of  Line  Objects  by  Structured  Parallel  Operations  34 

3.1  Structured  Image  Operations . 34 

3.2  Method  and  Results  of  Line  Extraction  . 35 

4  Structure  Oriented  Image  Analysis  with  Blackboard  Dyi  ,(>m  41 

4.1  Situation  Driven  and  Model  Controlled  Image  Analysis  ...  42 

4.1.1  Image  Preprocessing . 42 

4.1.2  Symbolic  Image  Analysis  . 42 

4.2  The  Blackboard  Production  System  . 47 

4.2.1  Blackboard . 47 

4.2.2  Production  Cycle . 47 

4.2.3  General  Assessment  and  Context  Spaces . 49 

4.2.4  Final  Assessment  and  Stop  Criterion . 51 

4.3  Edge  Oriented  Detection  of  Intersections . 51 

ii 


- -  —  ■» 


CONTENTS  iii 

4.3.1  Parallel  Rewrite  System . 51 

4.3.2  Set  of  Productions . 52 

4.3.3  Results  from  Detection  of  Intersections . 57 

5  Iconic  Image  Filtering  and  Segmentation  76 

5.1  Gray  Level  Pyramid . 76 

5.1.1  Generation  of  the  Gray  Level  Pyramid . 77 

5.1.2  Projection  of  an  Image  Point . 80 

5.2  Contrast  Pyramids . 80 

5.2.1  Detection  of  Edge  Elements . 80 

5.2.2  Detection  of  Spots . 81 

5.3  Edge  Oriented  Segmentation  . 84 

5.3.1  Starting  Point  List  for  Edge  Segmentation . 87 

5.3.2  Tracking  of  Edge  Elements  and  Feature  Extraction . 87 

5.3.3  Assessment  of  Edge  Elements  and  Segments . 89 

5.3.4  Approximation  of  Edge  Segments  by  Polygons  . 91 

5.3.5  Symbolic  Description  of  Edge  Segments . 93 

5.4  Area  Oriented  Segmentation . 93 

5.4.1  Starting  Point  List  for  Area  Segments . 95 

5.4.2  Spot  Projection  into  Levels  of  Higher  Resolution . 95 

5.4.3  Area  Segmentation  by  Binarization . 97 

5.4.4  Tracking  of  Contour  Elements  and  Feature  Extraction  ...  98 

5.4.5  Assessment  of  Area  Segments . 98 

5.4.6  Approximation  of  Area  Segments  by  Polygons . 100 

5.4.7  Symbolic  Description  of  Area  Segments . 107 

5.5  Results  from  Edge  and  Area  Segmentation . 107 

6  Extended  Production  Model  for  Street  Network  Extraction  109 

6.1  Area  Oriented  Detection  of  Intersections . Ill 

6.2  Extended  Set  of  Productions  . 112 

6.3  Results  from  Area  Oriented  Extraction  of  Street  Networks . 115 

6.3.1  PHOENIX-2A1 . 117 

6.3.2  PH0ENIX.2B1 . 125 

6.3.3  BIETIGHEIM1  _31 . 125 

6.3.4  BIETIGHEIM2.31 . 125 

7  Conclusions  147 

Acknowledgement  150 

Bibliography  151 


List  of  Figures 

1.1  Test  image  PHOENIX .  6 

1.2  Test  image  BIETIGHEIM1  .  7 

1.3  Test  image  BIETIGHEIM2 .  8 

2.1  Line  extraction  from  PHOENIX-1A/11/2/4/2/10/20/0  18 

2.2  Line  extraction  from  PHOENIX. 1A/22/2/4/2/10/20/0  19 

2.3  Line  extraction  from  PHOENIX.2A/11/2/4/2/10/20/0  20 

2.4  Line  extraction  from  PHOENIX.2A/22/2/4/2/10/20/0  21 

2.5  Line  extraction  from  BIETIGHEIM1.1/11/2/4/2/10/20/0 . 22 

2.6  Line  extraction  from  BIETIGHEIMl.1/11/2/4/2/10/20/20  ....  23 

2.7  Line  extraction  from  BIETIGHE1M1.2/11/2/4/2/10/20/0 . 24 

2.8  Line  extraction  from  BIETIGHEIM1.3/11/1/5/3/5/10/0  . 25 

2.9  Line  extraction  from  BIETIGHEIM2. 1/11/2/4/2/10/20/0 . 26 

2.10  Line  extraction  from  BIETIGHEIM2.1/11/2/4/2/5/20/0  . 27 

2.11  Line  extraction  from  BIETIGHEIM2-1/1 1/4/4/2/10/20/0 . 28 

2.12  Line  extraction  from  BIETIGHEIM2.2/1 1/4/5/3/10/20/0 . 29 

2.13  Line  extraction  from  BIETIGHEIM2.3/11/1/5/3/10/20/0 . 30 

2.14  Line  extraction  from  PHOENIX.1B/22/2/4/2/10/20/0  .  31 

2.15  Line  extraction  from  PHOENIX-2B/22/2/4/2/10/20/0  .  32 

2.16  Superimposed  dark  line  extraction  from  PHOENIX.2B  (/) . 33 

3.1  Image  processing  with  structured  parallel  operations . 35 

3.2  Estimates  of  sequential  and  parallel  image  processing . 36 

3.3  Superimposed  dark  line  extraction  from  PHOENIX.2B  {II)  ....  38 

3.4  Superimposed  dark  line  extraction  from  PHOENIX_2B  {III)  ...  39 

3.5  Superimposed  bright  line  extraction  from  PHOENIX.2B  {IV)  ...  40 

4.1  Preprocessing . 43 

4.2  Derivation  tree:  bridge . 44 

4.3  Structure  analysis  (/)  45 

4.4  Structure  analysis  {II) . 46 

4.5  Blackboard  system . 48 

4.6  Principle  of  a  production  cycle  . 49 

4.7  Example  of  context  space . 50 

4.8  Production  net . 53 

4.9  Principle  of  line  prolongation . 54 


IV 


LIST  OF  FIGURES 


V 


4.10  Test  image  PHOENIX.2A1  . 60 

4.11  Preprocessing  of  PHOENIX.2Al/base  objects . 61 

4.12  Analysis  of  PHOENIX_2Al/triggering  base  objects . 62 

4.13  Analysis  of  PHOENIX.2Al/extracted  objects . 63 

4.14  Test  image  PH0ENIX_2B1  . 64 

4.15  Preprocessing  of  PHOENIX.2Bl/base  objects . 65 

4.16  Analysis  of  PHOENIX.2Bl/triggering  base  objects . 66 

4.17  Analysis  of  PHOENIX.2Bl/extracted  objects . 67 

4.18  Test  image  BIETIGHEIM1 .31 . 68 

4.19  Preprocessing  of  BIETIGHEIM1  _31/base  objects . 69 

4.20  Analysis  of  BIETIGHEIM1 -31/triggering  base  objects . 70 

4.21  Analysis  of  BIETIGHEIMl-31/extracted  objects . 71 

4.22  Test  image  BIETIGHEIM2.31 . 72 

4.23  Preprocessing  of  BIETIGHEIM2_31/base  objects . 73 

4.24  Analysis  of  BIETIGHEIM2_31/triggering  base  objects . 74 

4.25  Analysis  of  BIETIGHEIM2_31/extracted  objects . 75 

5.1  Pyramidal  image  structure . 78 

5.2  Generation  of  a  pyramidal  image  point . 78 

5.3  Gray  level  pyramid  of  PHOENIX.2A1 . 79 

5.4  Absolute  value  of  gradients  of  PHOENIX.2A1 . 82 

5.5  Direction  of  gradients  of  PHOENIX.2A1 . 83 

5.6  Spot  curvature  pyramid  of  PHOENIX.2A1  . 85 

5.7  Spot  surroundedness  pyramid  of  PHOENIX. 2A1  86 

5.8  Coding  of  direction . 88 

5.9  Tracking  of  edge  elements . 88 

5.10  Ranking  of  successors  . 90 

5.11  Ranking  of  predecessor . 90 

5.12  Edge  oriented  segmentation  of  PHOENIX.2A1  . 92 

5.13  Symbolic  description  of  edge  segments . 94 

5.14  Projection  of  spots  into  levels  of  higher  resolution . 96 

5.15  Contour  tracking . 99 

5.16  Area  oriented  segmentation  of  PHOENIX.2A1  (bright  areas  I)  .  .  101 

5.17  Area  oriented  segmentation  of  PHOENIX.2A1  (bright  areas  II)  .  .  102 

5.18  Area  oriented  segmentation  of  PHOENIX  _2A1  (dark  areas  I)  .  .  .  103 

5.19  Area  oriented  segmentation  of  PH0ENIX.2A1  (dark  areas  II)  .  .  104 

5.20  Area  oriented  segmentation  of  PHOENIX.2A1  (bright  areas  III )  .  105 

5.21  Area  oriented  segmentation  of  PH0ENIX.2A1  (dark  areas  III)  .  .  106 

5.22  Symbolic  description  of  area  segments . 108 

6.1  Edge  oriented  analysis  of  256  x  256  section  of  PHOENIX.2A1  .  .  .  110 

6.2  Area  oriented  production  net . Ill 

6.3  Preprocessing  of  PHOENIX.2Al/small  areas . 118 

6.4  Analysis  of  PHOENIX_2Al/big  araes . 119 

6.5  Analysis  of  PHOENIX.2Al/long  areas . 120 


LIST  OF  FIGURES 


vi 

6.6  Analysis  of  PH0ENIX-2Al/streets . 121 

6.7  Analysis  of  PHOENIX  2Al/intersections . 122 

6.8  Analysis  of  PH0ENIX-2Al/street  networks . 123 

6.9  Analysis  of  PH0ENIX_2Al/extraction  results . 124 

6.10  Preprocessing  of  PHOENIX_2Bl/small  areas  . 126 

6.11  Analysis  of  PH0ENIX.2Bl/big  araes . 127 

6.12  Analysis  of  PHOENIX.2Bl/long  areas . 128 

6.13  Analysis  of  PH0ENIX-2Bl/streets . 129 

6.14  Analysis  of  PHOENIX_2Bl/intersections . 130 

6.15  Analysis  of  PHOENIX_2Bl/street  networks . 131 

6.16  Analysis  of  PHOENIX_2Bl/extraction  results . 132 

6.17  Preprocessing  of  BIETIGHEIM1 -31 /small  areas . 133 

6.18  Analysis  of  BIETIGHEIMl-31/big  araes . 134 

6.19  Analysis  of  BIETIGHEIM1.31/long  areas . 135 

6.20  Analysis  of  BIETIGHEIMl-31/streets . 136 

6.21  Analysis  of  BIETIGHEIM1  -31  /intersections . 137 

6.22  Analysis  of  BIETIGHEIMl-31/street  networks . 138 

6.23  Analysis  of  BIETIGHEIMl-31/extraction  results . 139 

6.24  Preprocessing  of  BIETIGHEIM2_31/small  areas . 140 

6.25  Analysis  of  BIETIGHEIM2-31/big  araes . 141 

6.26  Analysis  of  BIETIGHEIM2-31/long  areas . 142 

6.27  Analysis  of  BIETIGHEIM2-31/streets . 143 

6.28  Analysis  of  BIETlGHEIM2.31/intersections . 144 

6.29  Analysis  of  BIETIGHEIM2-31/street  networks . 145 

6.30  Analysis  of  BIETIGHEIM2-31/extraction  results . 146 


Summary 


The  work  can  be  summarized  as  follows: 

•  Application  of  a  sequential  iconic  image  processing  method  using  stream 
following  techniques  and  a  parallel  iconic  image  processing  method  us¬ 
ing  structured  parallel  operations  (SIO),  previously  developed  for  rural 
scenes,  to  urban  areas  demonstrating  the  need  for  more  powerful  image 
analyzing  methods. 

•  Application  of  an  existing  blackboard  oriented  image  analysis  (BPI) 
method  to  urban  areas  to  extract  intersections  with  streets  composed  of 
line  primitives  generated  from  existing  preprocessing. 

•  Development  of  a  method  for  segmenting  an  image  into  edge  and  area  el¬ 
ements  as  primitives  in  order  to  improve  the  preprocessing  for  blackboard 
oriented  analysis. 

•  Development  of  a  preliminary  model  for  the  extraction  of  the  traffic  net¬ 
work  of  streets  and  intersections  within  urban  areas  by  blackboard  ori¬ 
ented  image  analysis  (BPI)  and  development  of  an  interface  accepting 
area  segments  as  primitives  received  from  the  improved  preprocessing. 

•  Implementation  of  methods  in  FORTRAN  and  PASCAL  programming 
languages  in  a  VMS  operating  system  for  DEC  VAX  computers. 

•  Documentation  of  test  results  from  processing  suburban  imagery  of 
Phoenix,  Arizona  (USA)  and  Bietigheim,  Southern  Germany  (FRG)  pro¬ 
jected  with  a  scale  of  1:4,000  to  1:56,000  and  digitized  with  an  aperture 
of  400  fim  to  50  /im  representing  a  pixel  size  of  approximately  2.8  m  to 
0.4m  on  earth. 


Chapter  1 
Introduction 


Several  different  methods  for  the  automatic  extraction  of  line  objects  from 
aerial  imagery  of  rural  scenes,  based  on  differing  principles,  had  been  developed 
and  implemented  at  FIM  partly  in  charge  of  the  U.S.  Government  [2,8,12],  The 
methods  perform  the  extraction  task  conservatively,  e.g.  line  segments  which 
are  distorted  by  a  considerable  amount  of  noise  or  occlusion,  are  not  accepted. 
Thus  the  extraction  results,  e.g.  the  polygons  representing  the  network  of  line 
objects,  tend  to  be  correct,  reliable,  but  not  necessarily  complete. 

The  methods  [2]  had  been  extended  to  perform  the  extraction  task  in  two 
images  of  a  stereo  pair  [8],  The  extraction  results  of  this  process  have  a  higher 
degree  of  correctness  and  reliability,  because  they  are  produced  by  coincident 
extraction  from  two  images  of  the  same  area.  Procedures  for  automatic  ex¬ 
traction  of  line  shaped  objects  from  aerial  images  had  been  improved  and  com¬ 
pleted  by  artificial  intelligence  aspects  [12].  A  general  model  of  road  network 
had  been  used  to  complete  road  extraction  from  images.  Digital  elevation  data 
had  been  used  to  guide  the  process  of  river  and  creek  extraction  from  images. 


1.1  Problems 

The  basic  idea  for  the  automatic  extraction  of  line  objects  is  not  to  process 
and  analyze  the  complete  image  matrix  systematically  by  applying  a  variety  of 
special  operations.  The  resulting  multitude  of  preliminary  results  would  have 
to  undergo  complicated  compatibility  tests  which  will  -  in  most  cases  -  be  any¬ 
thing  else  but  trivial.  On  the  contrary,  the  process  is  focussed  only  on  promis¬ 
ing  sections  of  the  image  matrix  by  applying  locally  selective  hypothesis  and 
verification  techniques. 

Concerning  rural  scenes  the  solution  of  the  problem  had  been  attacked  in 
two  different  functional  steps  [2]: 

•  To  initialize  any  automatic  image  analysis  process,  some  sort  of  a  sys¬ 
tematic  screening  of  the  image  matrix  is  necessary.  As  roads  and  streets 
are  to  be  extracted  the  initial  screening  of  the  image  matrix  consists  of  a 
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search  for  line  shaped  segments  which  serve  as  starting  segments  or  cues 
for  an  extraction  process  (parallel  approach). 

•  It  is  supposed  that  the  search  for  starting  segments  of  lines  yields  results 
of  high  confidence.  This  is  the  case,  if  the  area  covered  by  the  image  is 
rural,  or  extends  from  urban  regions  into  neighboring  rural  country  or  at 
some  locations  in  urbanized  regions  where  undistorted  segments  of  roads 
are  clearly  detectable.  Beginning  at  those  starting  segments,  the  line  ex¬ 
traction  process  is  applied,  which  consists  of  an  alternating  locally  or  re¬ 
gionally  adaptive  gray  level  analysis  by  stream  following  techniques  (se¬ 
quential  approach). 

Transparencies  at  a  scale  of  appoximately  1:70,000  were  chosen  as  test  imagery 
digitized  by  a  scanning  aparature  of  25/rm  corresponding  to  a  circular  spot  size 
of  1.75  m  diameter  on  earth.  Thus  the  line  extraction  procedure  had  been 
adapted  to  a  line  width  of  about  3  to  8  pixels  only  corresponding  with  the 
width  of  roads  at  that  scale.  Later  on  a  new  problem  solution  of  the  second 
step  of  that  line  extraction  method  had  been  investigated  and  implemented 
by  a  parallel  approach  [3,9j.  The  parallel  approach  avoids  the  first  functional 
step  and  shows  nearly  equal  performance  as  the  sequential  approach.  The  ex¬ 
traction  of  line  objects  is  accomplished  by  iconic  structured  parallel  operations 
(SIO).  In  addition,  the  execution  time  of  different  parallel  processing  schemes 
had  been  estimated. 

When  applied  to  imagery  of  rural  scenes,  the  performance  of  such  an  auto¬ 
matic  line  extraction  method  can  be  considered  as  successful.  However,  when 
applied  to  imagery  of  urban  areas,  the  success  and  reliability  of  that  method 
decrease  remarkably  due  to  the  complex  mixture  of  natural  and  man  made  ob¬ 
jects  at  these  locations.  Because  of  the  complex  gray  level  situation  in  urban 
areas  the  locally  or  regionally  restricted  gray  level  analysis,  using  cues  of  arte¬ 
rial  roads  at  the  boundaries  from  rural  country  to  urban  areas,  e.g.,  will  soon 
fail  to  yield  reliable  detection  results  of  roads  and  streets. 

The  improvement  of  the  signal  processing  algorithms  without  incorporation 
of  semantics  does  not  produce  evident  better  results.  To  enhance  the  perfor¬ 
mance  of  the  extraction  procedure,  highly  sophisticated  knowledge  based  image 
analysis  is  necessary  to  allow  progress  in  this  research  field. 


1.2  New  Approach 

This  research  project  has  concentrated  on  the  detection  and  extraction  of  roads 
and  streets  in  the  context  of  surrounding  objects  in  images  of  urban  areas.  The 
work  was  started  applying  existing  image  processing  methods  for  segmentation 
and  feature  extraction  and  an  existing  knowledge  based  method  for  structure 
oriented  analysis  of  image  segments.  The  investigations  proceeded  by  improv¬ 
ing  the  preprocessing  and  adapting  the  structural  analysis.  Hence,  segmenta¬ 
tion  procedures  for  edge  like  and  area  like  objects  were  developed  and  a  first 
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simplified  model  for  the  dominant  structures  of  an  urbanized  area  and  a  con¬ 
trol  strategy  for  the  whole  procedure  were  considered  using  a  blackboard  ori¬ 
ented  approach  [5,6,9, 10,11].  Thereby,  only  area  like  objects  has  been  used  for 
structured  analysis  so  far. 

To  solve  at  least  part  of  automatic  line  network  extraction  problem  from 
imagery  of  urban  areas  the  application  of  a  compound  of  higher  sophisticated 
processing  methods  is  necessary.  The  basic  idea  of  the  new  approach  is  as  fol¬ 
lows: 

•  First,  the  gray  level  signal  analysis  has  to  be  extended  from  the  restricted 
locality  sufficient  for  the  sequential  stream  following  technique  to  a  larger 
area,  typically  512  x  512  pixels  as  for  the  parallel  approach.  Within  this 
area  the  gra"  level  signal  analysis  is  extended  to  detect  any  elementary 
segment,  e.g.  line  segments,  small  blobs,  small  areas,  etc.  Thereby  a 
distinct  initialization  step  as  used  for  the  stream  following  technique  is 
avoided. 

•  Next,  subsets  of  these  sets  of  elementary  segments  have  to  be  selected  on 
the  basis  of  similarity  of  properties  (features),  e.g.  line  segments  of  simi¬ 
lar  length,  orientation  and  distance.  It  has  been  proved  that  among  those 
subsets  of  elementary  segments  (primitives)  the  dominant  structures  of 
objects  can  be  found. 

•  Last,  a  comparison  to  a  detailed  model  of  the  relevant  structures  of  an 
urbanized  area  allows  the  classification  of  the  structures  four  -i  in  the  im¬ 
ages  and,  hence,  is  able  to  distinguish  roads  and  streets  from  other  dom¬ 
inant  objects  shown  in  the  image.  The  central  problem  consists  in  the 
development  of  a  suitable  model  of  an  urbanized  area.  The  model  is  or¬ 
ganized  in  the  form  of  a  blackboard  based  syntactical  production  system 
for  image  analysis  (BPI)  [11'.  The  contents  of  the  model  is  taken  from 
some  test  images.  Attention  has  been  given  to  the  problem  of  a  control 
strategy  for  a  suitable  cooperation  between  the  pool  of  (active)  image  an¬ 
alyzing  methods  and  the  (passive)  model  data. 

Thus,  feature  and  context  information  of  image  segmentation  received  from 
preprocessing  have  to  be  converted  from  raster  presentation  of  vectors  to  black¬ 
board  presentation  of  basic  element  ^primitives  or  base  objects)  for  structural 
analysis.  Attributes  are  associated  with  each  element  organized  in  sets  and 
stored  in  an  associative  memory.  The  structural  analysis  or  socalled  symbolic 
processing  is  carried  out  successively  on  symbols  (elements)  starting  with  basic 
ones  (primitives)  received  from  preprocessing. 

After  having  generated  new  symbols  according  to  a  production  net,  sym¬ 
bolic  processing  proceeds  with  deduced  symbols  (deduced  elements  or  partial 
objects).  For  effective  processing  of  interrelations,  the  symbol  (element  or  ob¬ 
ject)  space  with  its  attributes  can  be  matched  against  different  context  spaces 
containing  the  same  attributes.  From  thj  context  spaces  expectation  criteria 
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are  derived.  In  addition,  a  quality  measure  is  generated  for  basic  symbols  from 
preprocessing  and  for  each  deduced  symbol  from  symbolic  processing.  The  ex¬ 
pectation  criteria  and  the  quality  measures  representing  a  confidence  evaluation 
(symbol  or  object  assessment)  are  assigned  to  the  corresponding  symbols  (ele¬ 
ments  or  objects). 

The  process  of  symbolic  processing  (structural  analysis)  is  accomplished  sit¬ 
uation  driven  and  model  controlled.  This  means  that  the  next  step  of  symbolic 
processing  is  chosen  according  to  the  highest  evaluation  of  a  situation  compati¬ 
ble  with  the  model.  The  process  (recognition)  is  terminated  when  all  hypothe¬ 
ses  are  tested.  The  image  description  can  be  achieved  by  analyzing  the  nodes 
of  the  production  net  from  bottom  to  top.  The  top  nodes  reached,  e.g.  repre¬ 
sent  detected  objects.  The  blackboard  based  syntactical  production  system  for 
symbolic  processing  (BPI)  performs  processing  of  noisy  symbolic  image  data 
(elements  or  objects),  deduces  symbolic  data  from  vaguely  declared  knowledge, 
assigns  confidence  to  symbolic  data,  and  controls  execution  by  opportunistic 
scheduling  from  symbolic  data  situations. 

The  BPI  model  consists  of  a  blackboard  for  entering  symbolic  image  data 
(elements  or  objects),  a  number  of  modules  for  symbolic  processing  (structural 
analysis)  representing  knowledge  of  image  understanding  and  symbolic  process¬ 
ing  based  on  a  syntactical  production  system,  and  a  control  module.  The 
control  module  performs  recognition  of  the  best  data  situation  from  symbc' 
(element  or  object)  space,  assessment  and  assignment  of  appropriate  hypothe¬ 
ses  for  symbolic  processing  (situation  driven  and  model  controlled),  and  selec¬ 
tion  of  the  processing  modules  concerned  with  testing  the  assigned  hypotheses 
(structural  analysis). 

The  advantage  of  BPI  as  compared  with  other  concepts  is  its  structural 
flexibility  with  respect  to  the  extension  of  symbol  (element  or  object)  space, 
modularity  of  processing  (structural  analysis),  and  disposition  of  knowledge,  as 
well  as  its  capability  of  retaining  all  intermediate  steps  of  symbolic  processing 
and  avoiding  conclusive  decisions. 


1.3  Test  Imagery 

It  was  decided  to  select  test  imagery  of  two  levels  of  complexity  for  the  investi¬ 
gations  carried  out  during  this  research  project: 

•  Test  image  of  Phoenix,  Arizona  (USA),  where  housing  areas  follow  simple 
rules  and,  hence,  display  very  clear  structures  (low  level  of  complexity), 
and 

•  Test  images  of  Bietigheim,  Southern  Germany  (FRG),  where  the  urban 
structures  are  more  complicated  (medium  level  of  complexity). 


The  test  images  have  been  digitized  using  a  50  fim  or  100  nm  raster,  which 
results  in  a  pixel  size  of  approximately  0.4  m  to  1.0  m  on  earth.  The  chosen 
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original  image  matrices  have  a  size  of  4096  x  4096  respectively  2048  x  2048  pix¬ 
els.  In  some  cases  a  signal  processing  was  necessary  in  order  to  correct  scan 
errors  as  in  scenes  from  Bietigheim. 

In  order  to  test  the  line  extraction  methods  the  following  test  images  were 
used: 

•  PHOENIX  (black  white  positive,  see  Figure  1.1) 

4096  x  4096  pixels 

1  :  20,000  scale 

50  /J.m  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  BIETIGHEIM1  (black  white  positive,  see  Figure  1.2) 

4096  x  4096  pixels 

1  :  14,000  scale 

50  nm  pixel  size  (on  image) 

0.7  m  pixel  size  (on  earth) 

•  BIETIGHEIM2  (black  white  positive,  see  Figure  1.3) 

2048  x  2048  pixels 
1  :  4,000  scale 

100  fim  pixel  size  (on  image) 

0.4m  pixel  size  (on  earth) 

As  image  analysis  requires  access  to  different  resolution  levels  of  a  test  im¬ 
age  because  of  the  variable  size  of  the  segments  contained  in  every  image,  mul¬ 
tiresolution  images  were  generated  by  subsequent  averaging  of  the  gray  values 
of  four  adjacent  pixels.  This  results  in  a  series  of  image  matrices  (image  pyra¬ 
mids)  with  a  resolution  (or  size)  decreasing  by  a  factor  of  2  from  image  to  im¬ 
age. 


1.4  Implementations 

All  methods  have  been  implemented  by  experimental  software  for  the  purpose 
of  analyzing  images,  assessing  performance  and  limitations  of  algorithms,  and 
documenting  results.  FORTRAN  and  PASCAL  programming  languages  in  a 
VMS  operating  system  for  DEC  VAX  computers  have  been  used.  Software  for 
image  segmentation  for  edge  and  area  like  objects  has  been  developed  as  fol¬ 
lows: 


-  iconic  image  filtering  (preprocessing): 


* k  \vm 
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Figure  1.2:  Test  image  BIETIGHEIMl 

(original  matrix  size  4096  x  4096  pixels,  pixel  size  and  scale  50 /xm.  :  0.7m) 
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Figure  1.3:  Test  image  BIETIGHEIM2 

(original  matrix  size  2048  x  2048  pixels,  pixel  size  and  scale  100/rm  :  0.4m) 


CHAPTER  1.  INTRODUCTION 


9 


-  generation  of  a  gray  level  multiresolution  image  pyramid 

-  application  of  a  gradient  operator  to  detect  edge  elements 

-  application  of  a  spot  operator  to  cue  dark  and  bright  contrasting  ar¬ 
eas  in  pyramidal  levels  of  different  resolution 

-  iconic  image  segmentation  (preprocessing): 

-  tracking  of  edge  elements  and  combination  to  edge  like  objects 

-  thresholding  of  dark  and  bright  contrasting  areas  and  determination 
of  area  like  objects  by  contour  following 

-  feature  extraction  and  output  of  symbolic  description  of  edge  like 
objects 

-  feature  extraction  and  output  of  symbolic  description  of  area  like  ob¬ 
jects 

The  following  BPI  software  was  applied  to  symbolic  structure  oriented  im¬ 
age  analysis: 

-  BPI  precompiler  converting  BPI  source  code  of  knowledge  sources  to 
standard  PASCAL  code,  calling  BPI  runtime  system 

-  BPI  runtime  system 

-  queue  manager  for  priority  controlled  activation  of  knowledge 
sources 

-  associative  memory  simulator  for  blackboard  access  by  knowledge 
sources 

-  support  routines  for  handling  of  workstation  interactions  and  files 

That  BPI  software  has  not  been  developed  under  this  contract. 


1.5  Investigations 

The  investigations  being  conducted  during  the  period  of  the  contract  are  de¬ 
scribed  in  the  following  chapters  of  this  report.  The  work  can  be  summarized 
as  follows: 

•  Application  of  a  previously  developed  sequential  iconic  image  processing 
method  for  rural  scenes  to  urban  areas  demonstrating  the  need  for  more 
powerful  image  analyzing  methods.  That  method  could  be  used,  however, 
for  the  cueing  of  streets  by  extracting  arterial  roads  at  the  boundary  from 
rural  country  to  urban  areas.  Some  extraction  results  and  the  problems 
arising  from  misapplication  are  demonstrated  in  chapter  2. 
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•  Application  of  a  previously  developed  parallel  iconic  image  processing 
method  for  rural  scenes  to  urban  areas.  After  parameter  adaption  that 
line  extraction  method  using  structured  parallel  operations  (S10)  could 
be  used  for  the  extraction  of  roads  and  streets  from  both  rural  country 
and  urban  areas.  Some  extraction  results  are  demonstrated  in  chapter  3. 

•  Application  of  an  existing  blackboard  oriented  image  analysis  method  to 
urban  areas  to  extract  intersections  with  streets  composed  of  line  primi¬ 
tives  generated  from  existing  preprocessing.  That  method  could  be  used 
for  the  cueing  of  streets  in  the  urban  area  for  the  post  processing  of 
streets.  The  principles  of  the  method  together  with  extraction  results  are 
explained  in  chapter  4. 

•  Development  of  a  method  for  segmenting  an  image  into  edge  and  area  el¬ 
ements  as  primitives  in  order  to  improve  the  preprocessing  for  blackboard 
oriented  analysis.  The  method  and  the  results  obtained  from  preprocess¬ 
ing  are  described  in  chapter  5. 

•  Development  of  a  preliminary  model  for  the  extraction  of  the  traffic  net¬ 
work  of  streets  and  intersections  within  urban  areas  by  blackboard  ori¬ 
ented  image  analysis  and  development  of  an  interface  accepting  area  seg¬ 
ments  as  primitives  received  from  the  improved  preprocessing.  The  used 
model  and  the  results  obtained  with  this  new  approach  are  demonstrated 
in  chapter  6. 

Concluding  remarks  about  overall  performance,  remaining  problems  and  re¬ 
quirements  of  further  improvement  are  given  in  chapter  7. 


Chapter  2 

Extraction  of  Line  Objects  by 
Stream  Following 


A  method  for  the  automatic  extraction  of  roads  and  rivers  from  large  aerial  im¬ 
ages  covering  rural  scenes  had  been  developed  and  demonstrated  successfully 
during  previous  research  projects  for  USAETL  in  1981,  1983  and  1986  [2,8,12]. 
When  starting  this  project  that  method  had  been  applied  to  urban  scenes  first 
in  order  to  get  a  basis  for  comparison  of  the  old  method  with  the  new  ap¬ 
proach. 


2.1  The  Line  Extraction  Method 

The  line  extraction  method  using  stream  following  techniques  is  to  be  ex¬ 
plained  briefly.  The  method  consists  of  the  three  procedures: 

•  starting  point  search 

•  local  extraction  of  lines 

•  regional  extraction  of  lines 

The  procedure  for  starting  point  search  analyzes  sample  lines  or  columns 
of  the  image  to  detect  short  segments  of  line  objects,  the  features  of  which 
(width,  contrast,  straightness,  etc.  )  correspond  with  predefined  model  seg¬ 
ments. 

The  local  extraction  procedure  works  on  a  one  dimensional,  semicircular 
sample  line  taken  from  the  image  at  the  very  neighborhood  of  the  actual  pro¬ 
cessing  position;  the  gray  values  of  the  sample  line  are  analyzed  to  detect  a 
next  cross  section  of  the  line  object  which  agrees  with  the  cross  section  of  the 
actual  processing  position. 

The  regional  extraction  procedure  works  on  an  area  of  interett  the  location 
of  which  is  predicted  from  the  actual  processing  position.  The  gray  values  of 
that  area  are  analyzed  to  detect  several  collinearly  located  cross  sections  of  the 
line  object  which  can  be  accepted  as  a  continuing  segment  of  this  object. 
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The  combined  application  of  these  three  procedures,  starting  with  the  first 
one  and  alternating  the  second  and  third  one,  automatically  extracts  lines  from 
aerial  imagery.  Some  considerable  sophistication  had  been  integrated  into  the 
extraction  methods  to  tolerate  local  distortions  and  noise.  However,  the  extrac¬ 
tion  will  stop  in  all  cases,  where  the  decision  in  favor  of  the  line  continuation  is 
not  reliable.  At  these  locations,  a  variety  of  adjusting  steps  could  be  considered 
to  eliminate  extraction  stops  including  but  not  confined  to  the  following: 

•  changing  the  starting  and  processing  locations,  and 

•  changing  the  acceptance  thresholds  of  contrast  and  shape  features. 

More  details  about  the  line  extraction  methods  can  be  found  in  the  quoted  ref¬ 
erences  [2,8,12], 


2.2  Line  Extraction  from  Urban  Scenes 

The  procedures  for  the  detection  of  starting  points  and  for  the  subsequent  ex¬ 
traction  of  lines  from  rural  scenes  were  applied  to  selected  1024  x  1024  sections 
of  the  test  images  PHOENIX,  BIETIGHE1M1,  and  BIETIGHEIM2  with  sub¬ 
urban  scenes  consisting  of  mixed  rural  and  urban  areas. 

2.2.1  Test  Imagery 

The  following  images  were  used  for  test  purposes: 

•  PHOENIX-1  A  (1/4  section  of  PHOENIX) 

1024  x  1024  pixels 

1  :  40,000  virtual  scale 

100  pm  pixel  size  (on  image) 

2  m  pixel  size  (on  earth) 

•  PHOENIX-2A  (1/16  section  of  PHOENIX) 

1024  x  1024  pixels 
1  :  20,000  real  scale 
50 /ira  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  BIETIGHEIM1-1  (1/1  image  of  BIETIGHEIM1) 

1024  x  1024  pixels 
1  :  56,000  virtual  scale 
200  pm  pixel  size  (on  image) 


CHAPTER  2.  EXTRACTION  OF  LINE  OBJECTS 


2.8  m  pixel  size  (on  earth) 

•  BIETIGHEIM1.2  (1/4  section  of  BIETIGHEIM1) 

1024  x  1024  pixels 

1  :  28,000  virtual  scale 

100 /im  pixel  size  (on  image) 

1.4  m  pixel  size  (on  earth) 

•  BIETIGHEIM1.3  (1/16  section  of  BIETIGHEIM1) 

1024  x  1024  pixels 
1  :  14,000  real  scale 
50  fim  pixel  size  (on  image) 

0.7  m  pixel  size  (on  earth) 

•  BIET1GHE1M2-1  (2/1  composition  of  BIETIGHEIM2) 

1024  x  1024  virtual  pixels  (512  x  512  real  pixels) 

1  :  16,000  virtual  scale 
400 /zm  pixel  size  (on  image) 

1.6  m  pixel  size  (on  earth) 

•  BIETIGHEIM2-2  (1/1  image  of  B1ETIGHEIM2) 

1024  x  1024  pixels 
1  :  8,000  virtual  scale 
200  fim  pixel  size  (on  image) 

0.8  m  pixel  size  (on  earth) 

•  BIETIGHEIM2-3  (1/4  section  of  BIETIGHEIM2) 

1024  x  1024  pixels 
1  :  4,000  real  scale 
100  gm  pixel  size  (on  image) 

0.4  m  pixel  size  (on  earth) 

•  PHOENIX.IB  (1/4  section  of  PHOENIX) 

1024  x  1024  pixels 

1  :  40,000  virtual  scale 

100  fim  pixel  size  (on  image) 

2  m  pixel  size  (on  earth) 
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•  PH0ENIX-2B  (1/16  section  of  PHOENIX) 

1024  x  1024  pixels 
1  :  20,000  real  scale 
50  pm  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

2.2.2  Parameter  Adjustment 

The  road  extraction  procedure  can  be  adapted  to  a  scene  of  an  aerial  image  in 
accordance  with  scale  and  contrast  by  the  following  parameters  p,  to  p7: 

Pi)  kind  of  contrast  (bright  line  objects  coded  by  11  or  dark  line  objects  coded 
by  22) 

p2)  minimum  width  of  the  line  object  (default:  2  pixels) 

p3)  increment  for  minimum  width  representing  the  maximum  width  of  the  line 
object  (default:  4  pixels) 

p4 )  increment  for  maximum  width  representing  the  width  of  crossings  of  roads 
(default:  2  pixels) 

p5)  gray  value  difference  (default:  20),  fixing  the  gray  level  threshold  with  re¬ 
spect  to  the  gray  value  of  the  reference  pixel  characterizing  the  gray  value 
contrast  between  line  object  and  its  surrounding.  The  line  width  must 
fall  into  the  given  variation  width  of  minimum  (p2)  and  maximum  line 
width  (p2  +  Pi) 

pe)  absolute  variation  of  gray  value  for  the  next  standard  sample  line  charac¬ 
terizing  the  constancy  of  the  gray  values  within  the  line  object  (default: 
20) 

p7)  and  grid  distance  fixed  to  40  pixels  for  the  starting  point  search  variable 
only  by  shifting  the  processing  area  (default:  0  pixels) 

The  parameters  p\  to  p^  are  coded  by  their  number  of  pixels  or  their  gray 
value,  respectively. 

2.2.3  Results  of  Line  Extraction 

The  results  received  from  applying  the  line  extraction  method  to  the  test  im¬ 
agery  are  demonstrated  by  Figure  2.1  to  Figure  2.16.  The  extracted  lines  are 
overlaid  on  the  input  images  in  red  color.  The  parameter  settings  for  adjust¬ 
ment  are  assigned  by  appending  their  code  words  to  the  name  of  the  test  image 
as  name / p\! pi/  ...  /p7/.  The  test  results  in  detail  are: 

•  PHOENIX-1  A/1 1/2/4/2/10/20/0  (see  Figure  2.1) 


CHAPTER  2.  EXTRACTION  OF  LINE  OBJECTS 


15 


Figure  2.1  shows  a  section  of  the  test  image  PHOENIX  covering  a  sub¬ 
urban  area.  Only  few  bright  access  roads  and  other  line  objects, 
e.g.  along  the  water  way,  are  found  because  of  noise  from  the  ur¬ 
ban  structure  of  the  scene  and  misadjustment  of  parameters.  Within 
the  housing  area  no  streets  are  extracted. 

•  PHOENIX.1A/22/2/4/2/10/20/0  (see  Figure  2.2) 

Figure  2.2  shows  the  same  test  image  as  in  Figure  2.1,  however,  the  line 
extraction  method  is  adjusted  to  dark  line  objects.  The  line  extrac¬ 
tion  results  are  equally  poor  as  from  Figure  2.1. 

•  PHOENIX.2A/11/2/4/2/ 10/20/0  (see  Figure  2.3) 

Figure  2.3  shows  a  smaller  section  of  the  test  image  PHOENIX  than  in 
Figure  2.1,  whereby  the  line  extraction  method  is  adapted  to  smaller 
bright  line  objects.  Here,  lines  are  also  extracted  within  the  housing 
area,  however,  not  the  dark  streets  themselves  but  the  bright  side 
walks  are  found. 

•  PHOENIX.2A/22/2/4/2/10/20/0  (see  Figure  2.4) 

Figure  2.4  shows  the  same  test  image  as  in  Figure  2.3,  however,  the  line 
extraction  method  is  adjusted  to  dark  line  objects.  The  line  extrac¬ 
tion  result  within  the  housing  area  is  better  than  compared  with 
that  from  Figure  2.2,  however,  street  extraction  is  yet  incomplete 
because  of  noise  from  the  urban  structure  of  the  scene  and  missing 
flexibility  of  parameter  adjustment. 

•  BIETIGHEIM1 -1/1 1/2/4/2/10/20/0  (see  Figure  2.5) 

Figure  2.5  shows  the  test  image  BIETIGHEIM1  covering  rural  as  well  as 
suburban  areas  with  bright  roads  and  streets.  Depending  on  param¬ 
eter  adjustment  arterial  roads  can  be  extracted  from  the  rural  areas. 
Within  the  housing  areas  the  streets  are  partly  found. 

•  BIETIG HEIM  1-1/ 11/2/4/2/ 10/20/20  (see  Figure  2.6) 

Figure  2.6  shows  the  same  test  image  as  in  Figure  2.5,  however,  a  shift 
of  20  pixels  is  accomplished  with  respect  to  the  processing  area  with 
fixed  search  grid  in  order  to  see  the  influence  of  different  starting 
points.  The  line  extraction  result  is  not  identical  compared  with 
that  from  Figure  2.5  but  the  quality  of  road  and  street  extraction 
is  comparable. 


•  BIETIG  HEIM  1.2/ 11/2/4/2/ 10/20/0  (see  Figure  2.7) 
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Figure  2.7  shows  a  section  of  the  test  image  BIET1GHEIM1,  whereby 
the  line  extraction  method  is  adapted  to  smaller  bright  line  objects 
than  in  Figure  2.5.  Therefore  smaller  streets,  roads,  and  other  line 
objects  are  partly  found. 

•  BIETIGHEIM  1.3/1 1/1/5/3/5/10/0  (see  Figure  2.8) 

Figure  2.8  shows  a  smaller  section  of  the  test  image  BIETIGHEIM1 
than  in  Figure  2.7,  whereby  the  line  extraction  method  is  adapted 
to  even  more  variable  and  smaller  bright  line  objects  than  in  Fig¬ 
ure  2.5.  Therefore  only  some  borders  of  streets  and  other  line  ob¬ 
jects  are  partly  found. 

•  BIETIGHEIM2.1/1 1/2/4/2/10/20/0  (see  Figure  2.9) 

Figure  2.9  shows  the  test  image  BIETIGHEIM2  covering  another  scene 
with  rural  and  suburban  areas  with  bright  roads  and  streets.  If  the 
parameters  are  adjusted  well  the  arterial  roads  can  be  found  in  most 
cases.  Starting  with  the  arterial  roads  some  streets  in  the  urban  area 
are  found  in  the  close  prolongation  of  the  arterial  roads.  However, 
they  are  lost  when  penetrating  into  the  urban  area. 

•  BIETIGHEIM2.1/11/2/4/2/5/20/0  (see  Figure  2.10) 

Figure  2.10  shows  the  same  test  image  as  in  Figure  2.9.  Because  of 
lower  adjustment  of  parameters,  other  and  more  line  objects  are  pro¬ 
duced. 

•  BIETIGHEIM2-1/1 1/4/4/2/10/20/0  (see  Figure  2.11) 

Figure  2.11  shows  the  same  test  image  as  in  Figure  2.9.  Because  of  al¬ 
tered  adjustment  of  parameters,  other  line  objects  are  favored. 

•  BIETIGHEIM2 .2/ 11/4/5/3/ 10/20/0  (see  Figure  2.12) 

Figure  2.12  shows  the  test  image  BIETIGHEIM2,  where  the  line  extrac¬ 
tion  method  is  adapted  to  smaller  bright  line  objects  than  in  Fig¬ 
ure  2.9.  In  order  to  recover  the  streets  of  Figure  2.9  the  parameters 
Pi,p2,p3  have  to  be  increased.  Because  of  misadjustment  of  parame¬ 
ters  fewer  line  objects  are  found  than  in  Figure  2.9. 

•  BIETIGHEIM2-3/1 1/1/5/3/10/20/0  (see  Figure  2.13) 

Figure  2.13  shows  a  section  of  the  test  image  BIETIGHEIM2,  where  the 
line  extraction  method  is  adapted  to  even  smaller  bright  line  objects 
as  in  Figure  2.9.  Therefore  only  some  borders  of  streets  and  other 
line  objects  are  partly  found.  Because  of  misadjustment  of  parame¬ 
ters  fewer  line  objects  are  found  than  in  Figure  2.9. 
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•  PHOENIX-1B/22/2/4/2/10/20/0  (see  Figure  2.14) 

Figure  2.14  shows  another  section  of  the  test  image  PHOENIX  covering 
a  suburban  area.  Only  few  broad  dark  roads  and  other  line  ob  jects 
are  found  because  of  noise  from  the  urban  structure  of  the  scene  and 
misadjustment  of  parameters.  Within  the  housing  area  streets  are 
hardly  extracted. 

•  PHOENIX.2B/22/2/4/2/10/20/0  (see  Figure  2.15) 

Figure  2.15  shows  a  smaller  section  of  the  test  image  PHOENIX  than 
in  Figure  2.14,  whereby  the  line  extraction  method  is  adapted  to 
smaller  dark  line  objects.  Here,  lines  are  also  extracted  within  the 
housing  area,  however,  street  extraction  is  yet  incomplete  because  of 
noise  from  the  urban  structure  of  the  scene  and  missing  flexibility  of 
parameter  adjustment. 

In  general,  the  results  complied  to  what  was  expected:  The  reliability  of 
the  line  extraction  method  developed  for  rural  scenes  decreases  remarkly  when 
applied  to  imagery  of  urban  areas  due  to  the  complex  mixture  of  natural  and 
man  made  objects.  Outside  the  housing  areas  most  of  the  roads  can  be  found. 
However,  the  series  of  tests  show  that  the  method  of  line  extraction  is  not  ro¬ 
bust  when  applied  to  suburban  scenes.  The  resuits  are  very  sensitive  against 
the  parameter  settings  because  shape  and  contrast  of  streets  vary  strongly. 

Within  housing  areas  the  method  fails,  when  performing  the  line  extraction 
with  fixed  and  not  adapted  parameter  settings.  However,  when  superimpos¬ 
ing  many  results  extracted  with  different  parameter  settings  adapted  to  dif¬ 
ferent  resolution  levels  of  an  image  and  to  different  radiometric  and  geometric 
features  of  lines,  then  the  line  extraction  result  becomes  acceptable  as  demon¬ 
strated  by  Figure  2.16. 

•  PH0ENIX_2B/varying  parameters  (see  Figure  2.16)  with  superposition  of 
results  from 

PHOENIX-1  B/22/3/5/3/10/ 10/0 
PHOENIX-1B/22/3/5/3/10/10/10 
PHOENIX-2B/22/3/5/3/10/10/0 
PHOENIX.2B/11/3/5/3/10/10/0  (inverted  gray  values.) 

PHOENIX -2B/22/4/4/2/10/20/0 
PHOENIX.2B/22/4/5/3/ 10/10/0 
PHOENIX.2B/22/4/4/2/ 10/10/0 

Figure  2.16  shovs  a  section  of  the  test  image  PHOENIX  as  in  Fig¬ 
ure  2.15  with  the  line  extraction  method  adapted  to  dark  line  ob¬ 
jects.  In  case  of  superposition  of  the  seven  extraction  results  stated 
above,  the  dark  lines  within  the  housing  area  representing  most  of 
the  streets  are  extracted  nearly  completely. 
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Figure  2.3:  Line  extraction  from  PHOENIX.2A/11/2/4/2/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  50 pm  :  1  m) 
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Figure  2.4:  Line  extraction  from  PHOENIX-2A/22/2/4/2/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  5 Ofim  :  1  m) 
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Figure  2.6:  Line  extraction  from  BIETIGHEIM1-1/11/2/4/2/10/20/20 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  200^m  :  2.8m) 
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Figure  2.7:  Line  extraction  from  BIETIGHEIM1..2/11/2/4/2/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  100 fim  :  1.4m) 
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Figure  2.9:  Line  extraction  from  BIETIGHEIM2-1/11/2/4/2/10/20/0 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  400 (im  :  1.6m) 
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Figure  2.10:  Line  extraction  from  BIETIGHEIM2-1/11/2/4/2/5/20/0 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  400/im  :  1.6m) 
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Figure  2.11:  Line  extraction  from  BIETIGHEIM2.1/11/4/4/2/10/20/0 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  400^m  :  1.6 m) 
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Figure  2.12:  Line  extraction  from  BIETIGHEIM2-2/11/4/5/3/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  200 fim  :  0.8m) 
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Figure  2.13:  Line  extraction  from  BIETIGHEIM2-3/11/1/5/3/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  lOOjrm  :  0.4m) 
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Figure  2.14:  Line  extraction  from  PHOENIX-1B/22/2/4/2/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  lOO^m  :  2m) 
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Figure  2.15:  Line  extraction  from  PHOENIX-2B/22/2/4/2/10/20/0 
(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  50 fim  :  lm) 
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Chapter  3 

Extraction  of  Line  Objects  by 
Structured  Parallel  Operations 


In  chapter  2.  a  sequential  method  for  line  extraction  is  described  which  had 
been  developed  prior  to  this  project.  Later  on  a  new  problem  solution  of  that 
method  had  been  investigated  and  implemented  by  a  parallel  approach.  In  [3] 
it  is  suggested  to  formulate  algorithms  for  iconic  processing  by  structured  par¬ 
allel  operations  SIO.  Image  analysis  based  on  SIO  can  easily  be  transferred  to 
every  parallel  processing  architecture. 


3.1  Structured  Image  Operations 

The  model  for  context  dependent  parallel  processing  with  SIO  is  shown  in  Fig¬ 
ure  3.1.  A  SIO  is  performed  by  applying  a  function  /  on  image  data  i  to  pro¬ 
duce  output  data  o.  The  processing  P  of  the  function  /  is  controlled  by  4  de¬ 
scriptors  for  the  specification  of  functional  execution,  operational  area,  param¬ 
eters  adjustment,  and  operational  context.  Together  with  input  and  output  a 


SIO  is  defined  by  6  descriptors: 

0  ”  fsi,,1 

(*) 

with  SIO  =  \DI\DF\DN\DP\DC\DO\ 

and  descriptors  Dl 

input  data  list 

DF 

function  list /selection 

DN 

neighborhood  list /selection 

DP 

parameter  list/selection 

DC 

context  list/selection 

DO 

output  data  list 

If  structured  parallel  processing  is 

involved  in  methods,  it  is  easy  to  transfer 

the  algorithms  and  the  software,  e.g 

developed  on  general  purpose  comput- 
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Figure  3.1:  Image  processing  with  structured  parallel  operations  SIO  ([3]) 

ers,  to  special  parallel  computer  systems.  As  compared  with  computer  simula¬ 
tion  (VAX:  sequential  or  parallel/SIO  method)  speed  up  factors  of  5  orders  of 
magnitude  are  within  the  scope  (Figure  3.2),  when  using  special  systems  as  the 
flexible  image  processor  FLIP  (16  processor  cascade  see  j'4j),  a  multiple  instruc¬ 
tion  (MI)/single  data  (SD)  pipeline  processor,  a  single  instruction  (Sl)/multiple 
data  (MD)  cellular  processor,  or  a  multiple  instruction  (MI)/multiple  data 
(MD)  multiprocessor  system. 


3.2  Method  and  Results  of  Line  Extraction 

The  SIO  extraction  of  lines  from  rural  scenes  was  applied  to  a  selected  section 
of  the  test  image  PHOENIX  with  suburban  scenes  consisting  of  mixed  rural 
and  urban  areas.  The  following  image  was  used  for  test  purposes: 

•  PH0ENIX.2B  (1/16  section  of  PHOENIX) 

1024  x  1024  pixels 
1  :  20,000  real  scale 
50  fim  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

In  [3j  the  line  extraction  required  for  detecting  and  tracking  roads  in  aerial 
photos  had  been  performed  by  using  a  sequence  of  16  SIOs  with  the  following 
9  functions:  LOCAL  ADAPTIVE  THRESHOLD,  EDGE  DIRECTION,  DOU¬ 
BLE  EDGE,  LINE  LENGTH,  LINE  ENDINGS,  EDGE  AGGLOMERATION, 
REDUCE  LINE,  ENLARGE  LINE,  CONNECT  LINES.  Without  further  adap¬ 
tion  that  method  was  used  for  line  extraction  from  urban  imagery.  The  line 


CHAPTER  3.  STRUCTURED  PARALLEL  OPERATIONS 


36 


VAX 

sequential 

VAX 

SIO 

FLIP 

cascade 

MI/SD 

pipeline 

SI/MD 

128  x  128 

MI/MD 

64  x  64 

processing 

power 

1  MIPS 

1  MIPS 

45  MIPS 

50  MIPS 

10  MAOPS 

(array  operation*) 

processed 

data 

1  M  bytes 

8  M  bytes 

3.2  M  bytes 

totaj 

instructions 
or  array 
operations 

75  10* 

(30  mi  pet 
object  point) 

100  TO8 

75  TO* 

75  TO8 

0.4  TO3 

0.07  108 

processing 

time 

75  s 

100  s 

1.7  s 

1.5  s 

0.04  s 

0.007  s 

speed  up 
factor 

1 

0.75 

44 

48 

1,800 

10,700 

Figure  3.2:  Estimates  of  sequential  and  parallel  image  processing  [3] 


extraction  procedure  can  be  adjusted  to  a  scene  of  an  aerial  image  by  several 
parameters.  The  method  is  adjusted  to  the  detection  of  bright  lines  only.  In 
order  to  detect  dark  lines  the  gray  values  of  the  input  image  must  be  inverted 
prior  to  application  of  the  method.  Different  parameter  settings  were  applied 
using  only  the  four  parameters  Pi,p2,p3.P4  from  several  other  parameters: 

Pi)  kind  of  contrast  (bright  line  objects  coded  by  11  or  dark  line  objects  via 
image  inversion  coded  by  22) 

p2)  image  scaling  by  low  pass  filtering  of  the  image  of  given  format  (specifica¬ 
tion  by  number  of  pixels  per  edge  of  filtered  image) 

p3)  number  of  pixels  for  the  distance  of  the  center  of  the  search  area  for  anti 
parallel  edges 

Pt)  number  of  pixels  for  the  width  of  the  search  area  for  anti  parallel  edges 

The  results  received  from  applying  the  SIO  line  extraction  method  to  the  test 
imagery  are  demonstrated  by  Figure  3.3  to  Figure  3.5.  The  extracted  lines  are 
overlaid  on  the  input  image  in  red  color.  The  parameter  settings  for  adjust¬ 
ment  are  assigned  by  appending  their  code  words  to  the  name  of  the  test  image 
as  name/p-[/p2/p3/pi.  The  test  results  in  detail  are: 

•  PH0ENIX_2B/varying  parameters  (see  Figure  3.3)  with  superposition  of 
results  from 

PHOENIX.2B/22/512/3/4 

PHOENIX^2B/22/512/4/4 

Figure  3.3  shows  a  section  of  the  test  image  PHOENIX  as  in  Figure  2.16 
with  the  line  extraction  method  adapted  to  dark  line  objects.  In 
case  of  superposition  of  the  two  extraction  results  described  above, 
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the  dark  lines  within  the  housing  area  representing  most  of  the 
streets  are  extracted  nearly  completely. 

•  PHOENIX-2B/ varying  parameters  (see  Figure  3.4)  with  superposition  of 
results  from 

PHOENIX-2B/22/512/4/4 

PHOENIX-2B/22/1024/5.5/5 

Figure  3.4  shows  a  section  of  the  test  image  PHOENIX  as  in  Figure  3.3 
with  the  line  extraction  method  adapted  to  dark  line  objects  with 
greater  variation  of  line  width.  In  case  of  superposition  of  the  two 
extraction  results  described  above,  the  dark  lines  within  the  housing 
area  representing  most  of  the  streets  are  extracted  more  completely. 

•  PH0ENIX_2B/varying  parameters  (see  Figure  3.5)  with  superposition  of 
results  from 

PH0ENIX-2B/1 1/512/4.5/5 
PH0ENIX-2B/1 1/1024/3/4 

Figure  3.5  shows  a  section  of  the  test  image  PHOENIX  as  in  Figure  2.16 
with  the  line  extraction  method,  however,  adapted  to  bright  line  ob¬ 
jects.  In  case  of  superposition  of  the  two  extraction  results  described 
above,  the  bright  lines  within  the  housing  area  representing  border 
lines  of  the  streets,  ways,  and  other  line  objects  are  extracted  nearly 
completely. 


c 


Figure  3.3:  Superimposed  SIO  dark  line  extraction  from  PHOENIX-2B/ 

varying  parameters  /22/512/3/4  and  /22/512/4/4 

(matrix  size  1024  x  1024  pixels,  pixel  size  and  scale  50 fim  :  lm) 


CHAPTER  3.  STRUCTURED  PARALLEL 


r 


Chapter  4 

Structure  Oriented  Image 
Analysis  with  Blackboard 
System 


For  the  analysis  of  objects  in  images  a  syntactical  classification  scheme  is  used 
which  associates  each  object  with  a  reference  object  or  object  class  by  analyz¬ 
ing  the  structure  of  the  image  according  to  the  available  hypotheses.  A  spe¬ 
cial  type  of  such  classificator  had  been  realized  at  FIM  [9,11]  as  the  blackboard 
oriented  production  system  for  image  understanding  (BP1).  This  chapter  de¬ 
scribes  the  principle  of  the  structural  analysis  demonstrated  by  a  bridge  identi¬ 
fication  task.  In  addition  a  formerly  developed  model  for  the  detection  of  inter¬ 
sections  adapted  to  a  street  network  is  applied  to  sections  of  512  x  512  pixels 
from  a  subset  of  images  used  for  the  extraction  of  line  objects  by  stream  follow¬ 
ing: 

•  PH0ENIX.2A  (1/16  subimage  of  PHOENIX) 

1024  x  1024  pixels 
1  :  20,000  real  scale 
50  [im  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  PHOENIX.2B  (1/16  subimage  of  PHOENIX) 

1024  x  1024  pixels 
1  :  20,000  real  scale 
50  fun  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  BIETIGHEIM1.3  (1/16  subimage  of  BIETIGHEIM1) 

1024  x  1024  pixels 
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1  :  14,000  real  scale 
50  \im  pixel  size  (on  image) 

0.7  m  pixel  size  (on  earth) 

•  BIETIGHEIM2.3  (1/4  subimage  of  BIETIGHEIM2) 

1024  x  1024  pixels 
1  :  4,000  real  scale 
100  fim  pixel  size  (on  image) 

0.4  m  pixel  size  (on  earth) 

4.1  Situation  Driven  and  Model  Controlled 
Image  Analysis 

Starting  with  basic  elements  (terminals,  base  objects)  as  primitives  generated 
by  image  preprocessing,  the  BPI  system  produces  deduced  elements  (nontermi¬ 
nals,  partial  or  target  objects)  consisting  of  more  and  more  complex  objects  up 
to  the  target  object.  To  build  more  complex  objects  from  less  complex  ones, 
the  system  has  to  test  hypotheses  about  the  objects.  Each  hypothesis  is  tested 
by  a  knowledge  source  of  the  blackboard  system.  The  generated  objects  are 
stored  in  the  blackboard  memory  for  associative  access.  All  knowledge  sources 
are  used  in  parallel  to  get  all  possible  interpretations  of  the  image.  One  object 
can  be  part  of  several  more  complex  objects. 

4.1.1  Image  Preprocessing 

In  the  first  step  of  the  preprocessing  a  multilevel  binarization  controlled  by  his¬ 
togram  analysis  of  the  gray  value  image  is  accomplished.  Next  the  contour 
lines  received  by  this  method  are  approximated  by  straight  lines,  circle  lines,  or 
corners  as  primitives  (base  objects).  The  approximated  primitives  are  assessed 
according  to  their  quality  of  approximation  and  stored  in  the  blackboard.  They 
are  the  input  data  for  the  structural  analysis  by  symbolic  image  processing. 

In  Figure  4.1  a  gray  value  image  a)  is  compared  with  the  reconstructed  im¬ 
age  b)  built  up  by  the  contour  lines  which  are  approximated  by  straight  short 
lines  received  from  preprocessing.  The  picture  b)  presents  the  results  of  the 
preprocessing  and  gives  an  impression  of  the  input  data  used  for  symbolic  im¬ 
age  analysis:  The  number  of  the  gray  values  is  reduced  and  the  shape  of  the 
contours  of  areas  which  have  the  same  gray  value  is  simplified. 

4.1.2  Symbolic  Image  Analysis 

The  BPI  system  used  in  this  project  analyzes  the  structure  of  an  image  com¬ 
posed  of  the  primitives  (base  objects)  received  from  preprocessing.  Starting 
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a)  b) 

Figure  4.1:  Preprocessing  of  a  gray  value  image,  a)  original  gray  value  image, 
b)  image  reconstructed  by  contour  lines  approximated  by  pieces  of  lines 

with  these  base  objects  more  and  more  complex  partial  objects  (elements)  are 
deduced  according  to  the  available  hypotheses  and  the  presence  of  objects  in 
the  image  until  a  reference  object  or  an  object  (target  object)  of  a  given  class 
is  found  or  all  elements  are  processed.  A  derivation  tree  exists  for  each  gener¬ 
ated  reference,  resp.  target  object  with  the  root  of  it  being  the  reference,  resp. 
target  object  itself,  the  branches  being  the  deduced  objects,  and  the  leaves 
being  the  primitive  objects.  In  Figure  4.2  a  derivation  tree  is  shown  for  the 
reference  object  bridge.  Starting  with  the  base  objects  line  received  from  pre¬ 
processing  and  proceeding  with  the  partial  objects  long  line,  stripe,  and  road, 
resp.  freeway  ( left  side  and  right  side  from  the  bridge),  the  target  object  bridge 
is  constructed.  In  this  BPI  system,  for  example,  the  hypotheses  to  be  part 
of  a  road  and  to  be  part  of  a  freeway  left  or  right  of  the  bridge  are  assigned 
to  stripes.  The  knowledge  about  the  models  used  to  construct  the  objects  is 
stored  as  knowledge  sources  in  the  processing  modules. 

In  order  to  give  an  example,  the  results  of  structural  analysis  by  symbolic 
image  processing  (identification  of  a  bridge)  are  shown  in  Figure  4.3  and  Fig¬ 
ure  4.4.  In  the  parts  a)  and  c)  of  the  figures  all  elements  (partial  objects) 
generated  before  completing  the  identification  of  the  reference  object  bridge  are 
shown.  The  parts  b)  and  d)  show  only  those  elements  (partial  objects)  which 
belong  to  the  derivation  tree  of  the  reference  object  bridge. 
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c)  d) 

Figure  4.3:  Result  of  structure  analysis  bridge  (/),  a),  c)  generated  long  lines 
and  stripes,  b),  d)  long  lines  and  stripes  belonging  to  the  reference  object 
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a) 


b) 


c)  d) 

Figure  4.4:  Result  of  structure  analysis  bridge  [II),  a)  generated  parts  of  roads 
and  freeways,  b)  parts  of  roads  and  freeways  belonging  to  the  reference 
object,  c)  generated  bridge  positions,  d)  position  of  identified  bridge 
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This  BPI  system  had  been  used  successfully  to  identify  reference  objects 
like  a  bridge  or  an  intersection.  More  details  are  explained  in  [10].  For  an¬ 
other  project  the  BPI  system  was  extended  by  new  knowledge  modules  for  the 
classification  of  objects  especially  for  the  detection  of  intersections  (see  section 
4.3  [5,61). 

4.2  The  Blackboard  Production  System 

The  principle  structure  of  the  blackboard  system  used  for  identification  or  clas¬ 
sification  purposes  is  shown  in  Figure  4.5.  The  system  contains  a  memory 
(blackboard),  in  which  all  data  such  as  base  objects  (e.g.  line),  partial  objects 
(e.g.  road)  and  target  objects  (e.g.  bridge)  are  stored.  The  data  are  processed 
by  verification  programs  (processing  modules  with  knowledge  sources)  working 
independently  from  each  other.  The  data  transfer  (exchange)  may  occur  only 
via  the  blackboard. 

Every  blackboard  system  uses  a  supervisor  that  decides  which  data  (base  or 
partial  objects)  stored  in  the  blackboard  are  to  be  processed  next  and  which 
verification  program  must  become  active.  By  means  of  a  queue  and  by  using  an 
assessment  strategy  the  best  object  is  selected  for  every  processing  step. 

An  example  for  symbolic  image  processing  with  a  BPI  system  is  given  in 
the  Figures  4.3  and  4.4  shown  above. 

4.2.1  Blackboard 

The  blackboard  is  the  central  database  of  the  system  realized  as  associative 
memory  accessed  by  the  queue  manager  of  the  supervisor  and  by  all  pro¬ 
grams  of  the  processing  modules  for  testing  the  hypotheses.  Base  objects 
(e.g.  line),  partial  objects  (e.g.  parts  of  roads  and  freeways  as  long  line, 
stripe,  road,  free  way  left  and  freeway  right),  and  target  objects  (e.g.  bridge) 
called  objects  in  the  following  are  entered  into  the  blackboard.  Objects  are 
described  by  their  attributes.  Examples  for  attributes  are:  type,  length,  ori¬ 
entation,  or  position.  Therefore  an  object  of  the  database  could  be  character¬ 
ized  by  e.g.  type  4=  line,  length  4=  40,  orientation  in  degrees  4=  30,  and 
coordinates  of  the  gravity  point  4=-  (z  =  10,  y  =  50). 

4.2.2  Production  Cycle 

The  base  objects,  e.g.  short  lines  from  image  preprocessing  are  entered  as  as¬ 
sessed  elements  in  a  priority  driven  queue  for  symbolic  processing.  Furthermore 
they  are  stored  in  the  blackboard  together  with  their  description  by  attributes 
(Figure  4.6)  During  each  production  cycle  it  is  tried  to  combine  base  objects 
to  more  complex  partial  objects,  e.g.  long  lines,  or  to  compose  much  more 
complex  partial  objects,  e.g.  roads,  of  earlier  generated  partial  objects,  e.g. 
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Figure  4.6:  Principle  of  a  production  cycle 

stripes.  The  production  of  deduced  elements  proceeds  until  the  set  of  nonpro- 
cessed  elements  is  empty,  or  the  target  object,  e.g.  bridge  is  constructed. 

The  synthesis  of  a  more  complex  partial  object  from  a  less  complex  partial 
object  during  each  production  cycle  proceeds  in  two  steps.  Initially,  hypotheses 
are  assigned  to  objects  available  in  the  queue.  The  objects  are  priority  ordered 
according  to  their  assessment.  Finally,  the  hypotheses  assigned  to  the  objects 
are  tested,  and,  if  they  are  confirmed,  those  objects  allocated  to  a  confirmed 
hypothesis  are  generated,  assessed  and  put  into  the  blackboard  and  into  the 
queue. 

4.2.3  General  Assessment  and  Context  Spaces 

All  objects  not  yet  being  processed  are  put  into  the  queue.  It  is  the  main  task 
of  the  supervisor  to  select  from  the  probably  great  number  of  nonprocessed  ob¬ 
jects  those  objects,  which  have  a  good  assessment  with  respect  to  as  many  cri¬ 
teria  as  possible,  and  which,  therefore,  can  be  combined  properly  to  more  com¬ 
plex  objects. 

Base  objects  are  assessed  during  the  preprocessing  cycle  resulting  from  the 
quality  of  the  image  filtering  operation.  For  each  object  allocated  to  a  specific 
processing  module  (knowledge  source),  an  adequate  set  of  assessment  quanti¬ 
ties  has  to  be  defined  for  their  attributes.  First,  during  execution  time  of  the 
processing  module  when  generating  an  object  an  assessment  value  is  allocated 
to  each  assessment  quantity.  The  assessment  can  be  chosen  object  oriented  ac- 
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Figure  4.7:  Example  of  context  space  consisting  of  index  sets  for 
objects  and  context  each  addressable  by  attributes 

cording  to  the  quality  of  the  generation  or  application  oriented  according  to 
certain  search  criteria  for  object  identification  or  classification.  Second,  during 
the  control  cycle  of  the  supervisor  a  general  assessment  is  derived  from  the  val¬ 
ues  of  the  seperately  assessed  attributes  of  the  objects.  This  result  leads  to  the 
decision  which  object  is  to  be  selected  next  for  s;  '  bolic  processing. 

For  effective  processing  and  selection  of  objects,  the  object  space  with  its 
attributes  can  be  matched  against  different  context  spaces  containing  the  same 
attributes.  For  associative  access  the  objects  are  stored  in  a  relational  data 
base  using  index  lists  resident  in  memory.  The  index  sets  addressable  by  at¬ 
tributes  are  formatted  by  binary  coding  or  by  bit  masking  as  shown  in  Fig¬ 
ure  4.7.  Competitive  and  concurrent  knowledge  for  symbol  interpretation  is 
provided  by  context  being  at  the  disposal  of  the  supervisor.  The  context  space 
is  associated  to  the  object  space  as  shown  in  Figure  4.7.  In  [15]  a  proposal  for 
a  BPI  hardware  is  made. 
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4.2.4  Final  Assessment  and  Stop  Criterion 

An  additional  task  of  the  assessment  procedure  is  to  specify  the  quality  of  the 
target  object  with  which  a  reference  object  is  identified  or  an  object  of  a  class 
is  detected.  This  final  assessment  depends  on  the  degree  of  matching  between 
the  reference  model  and  the  target  object  (see  Figure  4.4c  and  4.4d).  There¬ 
fore  the  stop  criterion  for  an  identification  run  is  the  defined  degree  of  precision 
with  which  the  target  object  is  detected.  Generally,  in  case  of  no  reference 
object  can  be  identified  or  in  case  of  object  classification,  a  run  is  terminated 
after  all  objects  have  been  processed. 


4.3  Edge  Oriented  Detection  of  Intersections 

In  order  to  detect  intersections  a  formerly  developed  model  for  the  identifica¬ 
tion  of  crossing  roads  was  applied.  First  a  formal  description  of  the  BPI  system 
and  detailed  information  about  the  applied  productions  are  given.  Then  an  ex¬ 
ample  of  object  classification  by  blackboard  oriented  structural  image  analysis 
is  presented. 

4.3.1  Parallel  Rewrite  System 

Starting  with  primitives  received  from  preprocessing  more  and  more  complex 
partial  objects  are  generated  by  adequate  productions  (verification  programs) 
until,  if  present  in  the  image  to  be  analyzed,  the  desired  reference  object  or 
an  object  of  that  class  is  generated.  The  entire  process  can  be  described  as  a 
parallel  rewrite  system  R.  Referring  to  formal  languages  the  system  R  can  be 
expressed  as  a  4  tupel  of  sets,  however,  its  structural  complexity  of  elements 
(objects)  is  much  higher  than  known  from  string  generating  grammars  and  its 
deduced  elements  specified  by  productions  are  tested  inversely  for  recognition: 

R  =  (P’i,  S,  P)  with 

Vt  =  set  of  base  objects 
Vn  =  set  of  partial  objects 
S  =  set  of  target  objects 
P  —  set  of  productions 

The  objects  produced  by  preprocessing  can  be  considered  as  the  base  objects 
(e.g.  lines).  Partial  objects  are  all  objects  generated  by  the  productions  (e.g. 
long  lines,  stripes,  fields,  streets,  intersections).  The  productions  specify  the 
transformation  of  a  triggering  base  object  or  partial  object  together  with  other 
specified  objects  into  another  partial  object  or  target  object.  One  or  more  pro¬ 
ductions  exist  for  the  generation  of  each  object.  In  the  BPI  system  the  pro¬ 
ductions  are  realized  as  procedural  verification  programs.  The  target  objects 
(e.g.  street  network )  and  a  small  subset  of  those  objects  which  can  be  derived 
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from  the  targets  by  contain-the-part  relations  (e.g.  intersections)  are  the  image 
objects  to  be  detected  or  extracted,  respectively. 

4.3.2  Set  of  Productions 

The  following  definitions  are  applied  for  the  extraction  of  street  networks  with 
intersections: 

V,  =  {  line  } 

Vn  =  {  long  line,  stripe,  street,  field,  intersection  } 

5  =  {  field,  street  network  } 

P  =  {  P01:  line  (line)  — >  long  line; 

P 02:  long  line  (line,  long  line)  — *  long  line; 

P 03:  long  line  (line,  long  line)  — -»  stripe; 

P 04:  long  line  (line,  long  line)  — *  stripe,  stripe; 

P 05:  stripe  (stripe)  — >  street; 

P06:  stripe  (stripe)  — »  field; 

POT:  street  ( stripe ,  street)  — *  street; 

P 08:  street  (street)  — »  intersection,  ...,  intersection; 

P 09:  intersection  ( street ,  intersection )  — >  street  network; 

P 10:  street  network  (street  network)  — *  street  network  } 

The  BPI  system  had  been  originally  applied  to  identification  tasks  with  ref¬ 
erence  objects  strongly  described  by  their  attributes.  When  applying  the  BPI 
system  to  the  classification  of  objects,  resp.  detection  of  the  target  object,  the 
classes  must  be  desribed  weakly  by  the  attributes  of  the  objects  concerned. 
Therefore  additional  classes  below  and  above  the  class  to  be  recognized  have 
to  be  introduced  to  achieve  a  high  performance. 

In  order  to  reduce  misrecognition  of  streets,  for  example,  the  competing 
partial  object  field  below  the  object  class  intersection  is  introduced  because  of 
confusion  between  streets  and  fields.  And  above  the  object  class  intersection 
the  integrating  target  object  street  network  is  introduced  to  guarantee  that  only 
intersections  are  recognized  which  can  be  connected  to  a  street  network.  All 
detected  isolated  intersections  get  a  worse  assessment  than  that  intersections 
which  lie  within  the  street  network. 

Starting  from  a  base  object  the  i s-part-of  relations  of  the  production  set 
which  generally  are  not  free  of  context  can  be  described  graphically  as  a  pro¬ 
duction  net.  The  applied  production  net  for  the  object  class  street  network  with 
the  contextfree  productions  P01  to  P10  is  shown  in  Figure  4.8.  Here  an  arrow 
corresponds  to  the  contextfree  relation  is-part-of  and  the  production  net  can  be 
structured  hierarchically. 

The  productions  P01  to  P10  are  described  in  detail  in  the  following  subsec¬ 
tions.  Thereby  the  socalled  triggering  partial  object  denotes  that  object  which 
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Figure  4.8:  Production  net  for  the  edge  oriented  detection  of  intersections 
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Figure  4.9:  Principle  of  line  prolongation 

is  chosen  from  the  supervisor  as  the  (at  present)  best  object  for  further  process¬ 
ing  (see  Figure  4.6). 

P01:  Line  — *  Long  Line 

If  a  base  object  line  is  selected  as  best  object,  then  the  hypothesis  for  the  par¬ 
tial  object  long  line  has  to  be  tested.  The  associated  verification  program  for 
testing  that  hypothesis  is  triggered  by  the  base  object  line  selected  as  best  ob¬ 
ject.  For  testing  purpose  a  narrow  search  area  around  the  selected  line  is  ex¬ 
amined  in  order  to  detect  other  lines  with  a  similar  orientation  (parallelism). 
In  case  of  a  successful  examination  the  partial  object  long  line  is  built  up  by 
tne  selected  line  together  with  those  lines  found.  After  composition  of  the  long 
line  an  assessment  is  accomplished  according  to  the  overlapping  of  lines  and 
the  length  of  the  long  line.  Figure  4.9  shows  the  principle  of  line  prolongation. 
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P02:  Long  Line  — >  Long  Line 

If  a  partial  object  long  line  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  long  line  has  to  be  tested  in  addition  to  the  hypotheses  P 03 
and  P04.  The  associated  verification  program  for  testing  that  hypothesis  is 
triggered  by  the  partial  object  long  line  selected  as  best  object.  For  testing 
purpose  a  narrow  search  area  around  the  selected  long  line  is  examined  in  order 
to  detect  other  lines  or  long  lines  with  a  similar  orientation  (parallelism).  In 
case  of  a  successful  examination  the  partial  object  long  line  is  built  up  by  the 
selected  long  line  together  with  those  lines  and  long  lines  found.  After  compo¬ 
sition  of  the  long  line  an  assessment  is  accomplished  according  to  the  overlap¬ 
ping  of  lines  and  long  lines,  and  the  length  of  the  generated  long  line. 

P03:  Long  Line  - — >  Stripe 

If  a  partial  object  long  line  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  stripe  has  to  be  tested  in  addition  to  the  hypotheses  P02 
and  P 04.  The  associated  verification  program  for  testing  that  hypothesis  is 
triggered  by  the  partial  object  long  line  selected  as  best  object.  For  testing 
purpose  a  narrow  search  area  on  both  sides  of  this  long  line  is  examined  in 
order  to  detect  other  long  lines  with  a  similar  orientation.  The  width  of  the 
search  area  can  vary  within  given  limits.  In  case  of  a  successful  examination  for 
one  deduced  object  the  partial  object  stripe  is  built  up  by  the  selected  long  line 
together  with  those  lines  and  long  lines  found.  After  composition  of  the  stripe 
an  assessment  is  accomplished  according  to  the  length  of  the  generated  stripe 
and  the  accuracy  of  parallelism  of  long  lines. 

P04:  Long  Line  — >  Stripe,  Stripe 

If  a  partial  object  long  line  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  stripe  has  to  be  tested  in  addition  to  the  hypotheses  P02  and 
P03.  The  associated  verification  program  for  testing  that  hypothesis  is  trig¬ 
gered  by  the  partial  object  long  line  selected  as  best  object.  For  testing  pur¬ 
pose  a  narrow  search  area  on  both  sides  of  this  long  line  is  examined  in  order 
to  detect  other  long  lines  with  a  similar  orientation.  In  case  of  a  successful  ex¬ 
amination  for  a  pair  of  deduced  neighboring  objects  two  partial  objects  stripe 
are  built  up  by  the  selected  long  line  together  with  those  lines  and  long  lines 
found.  After  composition  of  the  two  stripes  tangent  to  each  other  an  assess¬ 
ment  is  accomplished  according  to  the  length  of  the  generated  stripes  and  the 
accuracy  of  parallelism  of  long  lines. 

P05:  Stripe  — *  Street 

If  a  partial  object  stripe  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  street  has  to  be  tested  in  addition  to  the  hypothesis  P06. 
The  associated  verification  program  for  testing  that  hypothesis  is  triggered  by 


CHAPTER  4.  STRUCTURE  ORIENTED  IMAGE  ANALYSIS 


56 


the  partial  object  stripe  selected  as  best  object.  For  testing  purpose  a  narrow 
search  area  around  this  element  is  examined  in  order  to  detect  other  stripes 
with  a  similar  orientation  for  stripe  prolongation.  In  case  of  a  successful  exam¬ 
ination  the  partial  object  street  is  built  up  by  the  selected  stripe  together  with 
those  stripes  found.  After  composition  of  the  street  an  assessment  is  accom¬ 
plished  according  to  the  length  of  the  generated  street  and  its  accuracy  from 
stripe  composition. 

POO:  Stripe  — ->  Field 

If  a  partial  object  stripe  is  selected  as  best  object,  then  the  hypothesis  for  the 
partial  object  field  has  to  be  tested  in  addition  to  the  hypothesis  P05.  The 
associated  verification  program  for  testing  that  hypothesis  is  triggered  by  the 
partial  object  stripe  selected  as  best  object.  For  testing  purpose  a  narrow 
search  area  around  this  element  is  examined  in  order  to  detect  other  stripes 
with  a  similar  orientation  for  stripe  prolongation.  In  case  of  a  successful  ex¬ 
amination  a  street  like  stripe  prolongation  is  built  up  by  the  selected  stripe 
together  with  those  stripes  found.  If  the  found  stripe  prolongation  has  many 
parallel  fictitious  “roadsides”  then  it  is  implied  that  the  street  like  object  is  a 
field.  Therefore  the  partial  object  field  is  generated  instead  of  the  partial  object 
street  .  The  composition  and  assessment  of  the  field  may  be  suppressed. 

P07:  Street  — >  Street 

If  a  partial  object  street  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  street  has  to  be  tested  in  addition  to  the  hypothesis  P08. 
The  associated  verification  program  for  testing  that  hypothesis  is  triggered  by 
the  partial  object  street  selected  as  best  object.  For  testing  purpose  a  narrow 
search  area  around  this  element  is  examined  in  order  to  detect  other  stripes  or 
streets  with  a  similar  orientation  for  street  prolongation.  In  case  of  a  successful 
examination  the  partial  object  street  is  built  up  by  the  selected  street  together 
with  those  stripes  and  streets  found.  After  composition  of  the  street  an  assess¬ 
ment  is  accomplished  according  to  the  length  of  the  generated  street  and  its 
accuracy  from  stripe  and  street  composition. 

P08:  Street  — >  Intersection,  Intersection 

If  a  partial  object  street  is  selected  as  best  object,  then  the  hypothesis  for  the 
partial  object  intersection  has  to  be  tested  in  addition  to  the  hypothesis  P07. 
The  associated  verification  program  for  testing  that  hypothesis  is  triggered  by 
the  partial  object  street  selected  as  best  object.  For  testing  purpose  many 
search  areas  originating  from  equidistant  positions  along  the  selected  street  are 
oriented  at  different  angles  with  respect  to  the  direction  of  the  street.  The  se¬ 
lected  street  is  used  to  find  other  streets  which  have  a  crossing  with  this  street 
within  the  image.  The  streets  found  have  to  meet  the  criterion  closeness  to  the 
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selected  street.  In  case  of  a  successful  examination  the  partial  objects  intersec¬ 
tion  are  built  up  by  the  selected  street  together  with  those  streets  found.  After 
composition  of  the  intersections  an  assessment  is  accomplished  according  to  the 
above  mentioned  criterion. 

P09:  Intersection  — *  Street  Network 

If  a  partial  object  intersection  is  selected  as  best  object,  then  the  hypothesis 
for  the  target  object  street  network  has  to  be  tested  in  addition  to  the  hypothe¬ 
sis  P10.  The  associated  verification  program  for  testing  that  hypothesis  is  trig¬ 
gered  by  the  partial  object  intersection  selected  as  best  object.  For  testing  pur¬ 
pose  all  streets  belonging  to  the  selected  intersection  are  examined  with  regard 
to  other  intersections  assigned  to  those  streets  or  to  edges  of  the  image.  In  case 
of  a  successful  examination  the  target  object  street  network  is  built  up  by  the 
selected  intersection  together  with  those  sections  of  the  streets  which  are  neigh¬ 
boring  to  the  selected  intersection.  If  no  neighboring  intersection  exists  that 
section  of  the  street  which  leads  to  the  edge  of  the  image  is  added  to  the  street 
network.  After  composition  of  the  street  network  the  assessment  of  all  intersec¬ 
tions  involved  are  increased  above  a  given  threshold.  However,  in  cases  that  the 
selected  intersection  cannot  be  integrated  into  the  street  network  its  assessment 
is  reduced.  If  the  intersection  can  be  integrated  into  the  street  network  later  on 
then  its  assessment  is  increased  again. 

P10:  Street  Network  — *  Street  Network 

All  partial  objects  intersection  generated  by  testing  the  hypothesis  P 09  are  as¬ 
sembled.  The  intersections  with  their  associated  streets  can  be  accessed  by 
searching  for  assessment  values  greater  than  the  threshold  given  for  successful 
examination  of  intersections. 

4.3.3  Results  from  Detection  of  Intersections 

In  order  to  test  the  model  for  the  edge  oriented  detection  of  intersections  de¬ 
scribed  in  the  previous  section  the  following  images  were  used: 

•  PHOENIX.2A1  (1/4  section  of  PHOENIX-2,  see  Figure  4.10) 

512  x  512  pixels 
1  :  20,000  real  scale 
50  fim  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  PHOENIX.2B1  (1/4  section  of  PHOENIX.2,  see  Figure  4.1 1) 

512  x  512  pixels 
1  :  20,000  real  scale 
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50  fJ.m  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  BIETIGHEIM1_31  (1/4  section  of  BIETIGHEIM1-3,  see  Figure  4.18) 

512  x  512  pixels 
1  :  14,000  real  scale 
50  p.m  pixel  size  (on  image) 

0.7  m  pixel  size  (on  earth) 

•  BIETIGHEIM2.31  (1/4  section  of  BIETIGHEIM2.3,  see  Figure  4.22) 

512  x  512  pixels 

1  :  4,000  real  scale 

100  fxm  pixel  size  (on  image) 

0.4  m  pixel  size  (on  earth) 

Figure  4.11,  4.15,  4.19,  and  4.23  show  the  gray  value  images  after  preprocess¬ 
ing.  The  overlaid  yellow  lines  are  the  base  objects  line  from  preprocessing. 
In  the  f.st  step  of  the  preprocessing  a  binarization  by  10  levels  controlled  by 
histogram  analysis  of  the  gray  value  image  is  accomplished.  Next  the  contour 
lines  received  by  this  method  are  approximated  by  straight  lines.  This  is  done 
by  fixing  a  start  point  and  counting  40  pixels  (stick  length)  on  the  contour  line 
to  yield  the  end  point.  Start  point  and  end  point  are  connected  by  a  straight 
line  and  both  the  area  between  straight  line  and  contour  line  and  the  distance 
between  start  point  and  end  point  are  determined.  From  both  measures  the 
quality  of  the  detected  line  primitive  of  given  stick  length  is  classified.  For  the 
assessment  of  the  line  primitive  6  quality  classes  are  provided.  Thereby  the 
sixth  class  represents  no  qualification  for  a  straight  line  and  the  first  class  an 
excellent  straight  line  with  respect  to  the  given  stick  length.  Depending  on  the 
line  quality  the  stick  length  may  be  varied  and  another  trial  may  be  accom¬ 
plished.  The  next  step  of  line  approximation  may  preceed  starting  from  a  new 
contour  point  placed  back  a  given  number  of  pixels,  e.g.  20  pixels  on  the  con¬ 
tour  line. 

In  Figure  4.12,  4.16,  4.20,  and  4.24  the  triggering  base  objects  line  are  pre¬ 
sented.  The  triggering  lines  are  displayed  in  yellow  color  denoting  those  objects 
which  at  any  given  time  are  chosen  from  the  supervisor  as  the  best  object  for 
further  processing  with  respect  to  the  production  E01. 

In  Figure  4.13,  4.17,  4.21,  and  4.25  the  results  of  the  detection  of  intersec¬ 
tions  are  presented.  The  object  class  interjection  is  marked  in  green  color.  In¬ 
tersections  are  connected  by  straight  thin  green  lines  in  order  to  indicate  the 
target  object  street  network.  The  partial  objects  street  are  displayed  in  red 
color,  whereas  the  noncomposing  partial  objects  stripe  and  long  line  are  dis¬ 
played  in  yellow  or  blue,  respectively. 


CHAPTER  4.  STRUCTURE  ORIENTED  IMAGE  ANALYSIS 


59 


Comparing  the  results  of  street  extraction  achieved  from  structure  oriented 
image  analysis  with  those  from  the  line  object  extraction  by  stream  following  a 
considerable  gain  in  recognition  performance  can  be  observed.  The  used  model 
works  quite  well  if  the  streets  have  no  curves.  This  is  due  to  the  fact  that  the 
model  uses  only  straight  lines.  With  a  better  parameter  setting  one  can  sup¬ 
pose  that  shorter  streets  are  found.  Obviously  a  more  detailed  image  prepro¬ 
cessing  together  with  a  more  sophisticated  symbolic  description  of  objects  for 
structural  image  analysis  are  required. 


Figure  4.10:  Test  image  PHOENIX.2A1 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 pm  :  Ira) 
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Figure  4.11:  Preprocessing  of  PH0ENIX-2A  1/base  objects:  line  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 /im  :  lm) 
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Figure  4.12:  Analysis  of  PH0ENIX-2A1  /triggering  base  objects:  line  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  lm) 


Figure  4.13:  Analysis  of  PH0ENIX_2Al/target  object:  street  network  (green) 
partial  objects:  intersection  (green),  street  (red), 
noncomposing  partial  objects:  stripe  (yellow),  long  line  (blue) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  lm) 
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Figure  4.14:  Test  image  PH0ENIX.2B1 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50/rm  :  lm) 
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Figure  4.15:  Preprocessing  of  PH0ENIX-2Bl/base  objects:  line  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  lm) 
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Figure  4.16:  Analysis  of  PH0ENIX-2Bl/triggering  base  objects:  line  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 pm  :  1  m) 


Figure  4.17:  Analysis  of  PH0ENIX_2Bl/target  object:  street  network  (green), 
partial  objects:  intersection  (green),  street  (red), 
noncomposing  partial  objects:  stripe  (yellow),  long  line  (blue) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  bO/xm  :  lm) 


j?r- 
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Figure  4.19:  Preprocessing  of  BIETIGHEIMl_31/base  objects:  line  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50^im  :  0.7m) 
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Figure  4.20:  Analysis  of  BIETIGHEIMl-31/triggering  base  objects: 

line  (yellow)  (matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  0.7m) 
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Figure  4.22:  Test  image  BIETIGHEIM2.31 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  100 fim  :  0.4m) 


Figure  4.23:  Preprocessing  of  BIETIGHEIM2,31/base  objects:  /me  (yellow J 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  100pm  :  0.4m) 
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Figure  4.24-.  Analysis  of  B1ET1GHEIM2  31/triggering  base  objects: 
line  (yellow)  (matrix  size  512  x  512  pixels,  pixel  size  and  scale  100pm  :  0.4 


Figure  4.25:  Analysis  of  BIETIGHElM2_31/target  object: 

street  network  (green),  partial  objects:  intersection  (green),  street  (red), 

noncomposing  partial  objects:  stripe  (yellow),  long  line  (blue) 


(matrix  size  512  x  512  pixels,  pixel  size  and  scale  lOO/rm  :  0.4m) 


Chapter  5 

Iconic  Image  Filtering  and 
Segmentation 


The  implemented  improvement  of  image  preprocessing  required  for  street  net¬ 
work  analysis  by  means  of  an  extended  structural  model  is  described  in  this 
chapter. 

First,  a  multiresolution  gray  level  pyramid  by  successive  reduction  of  the 
digitized  image  is  computed.  Then  two  image  filtering  operators,  which  trans¬ 
form  the  gray  values  into  gray  value  differences  representing  contrast  informa¬ 
tion  of  the  image,  are  applied  to  the  gray  level  pyramid.  The  first  operator 
generates  edge  elements  which  are  combined  to  edge  segments  in  a  next  step. 
The  second  operator  is  a  spot  detector  which  indicates  bright  or  dark  contrast¬ 
ing  spots  in  the  gray  level  pyramid.  These  spots  are  indicators  used  to  extract 
area  elements  from  the  high  resolution  level  of  the  pyramid.  These  two  pro¬ 
cedures  are  utilized  for  iconic  image  preprocessing  to  compute  area,  resp.  edge 
like  image  segments.  The  results  of  these  segmentation  procedures  are  symbol¬ 
ically  described  and  stored  in  a  blackboard  for  further  processing  by  structural 
image  analysis. 

The  preprocessing  is  applied  to  a  512  x  512  section  of  the  image 
PH0ENIX.2A  for  demonstration  of  the  implemented  methods: 

•  PH0ENIX.2A1  (1/4  section  of  PH0ENIX_2A) 

512  x  512  pixels 
1  :  20,000  real  scale 
50  fim  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

5.1  Gray  Level  Pyramid 

With  respect  to  the  observation  that  objects  often  can  be  more  easily  recog¬ 
nized  in  images  of  low  resolution  (this  is  due  to  the  fact  that  confusing  details 
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in  the  high  resolution  images  are  not  present  in  the  low  resolution  ones),  a  gray 
level  pyramid  of  the  image  has  to  be  generated  before  filtering  [13]. 


5.1.1  Generation  of  the  Gray  Level  Pyramid 

The  pyramidal  image  structure  consists  of  a  series  of  images  representing  differ¬ 
ent  resolution  levels  of  the  original  image.  In  order  to  generate  these  different 
resolved  images  the  image  scanned  with  the  best  resolution  is  filtered  by  a  two 
dimensional  low  pass  of  variable  smoothing  strength.  This  procedure  is  equiv¬ 
alent  to  a  repeated  scanning  with  different  sampling  rates.  Another  possibility 
to  generate  such  a  pyramid  is  to  combine  nonoverlapping  cells  of  m  x  m  pixels 
to  a  new  image  point  in  the  next  resolution  level  of  the  pyramid. 

Figure  5.1  shows  the  pyramidal  image  structure  realized  by  the  procedure 
described  as  follows.  The  original  image  is  divided  into  squares  with  the  size  of 
2x2  pixels.  Each  square  in  a  level  n  is  transformed  into  one  image  point  in  the 
level  7i  —  l.  This  process  is  repeated  for  each  level  of  the  pyramid  until  the  top 
of  it  is  reached.  Therefore  a  pyramid  consists  of  N  +  1  levels  with  the  dimension 
2"  x  2n,  with  7i  =  0,1,  ...  V.  The  level  with  the  dimension  2N  x  2N  is  the  basis 
of  the  pyramid,  whereas  the  top  of  the  pyramid  is  built  by  the  single  point 
(2°  x  2°)  of  the  level  0.  Consequently  each  level  of  the  pyramid  consists  of  a 
matrix  (s,j)*n\  with  i,j  =  1,2,  ...,2",  and  n  =  0,1,  ...,N.  When  transferring 
from  level  n  with  an  image  size  of  n  x  n  pixels  to  level  n  —  1,  the  image  size  is 
reduced  to  n/2  x  n/2  pixels,  that  is  a  quarter  of  the  original  image  size. 

The  computation  of  the  matrix  from  the  matrix  in  any 

level  k  is  done  by  a  suitable  operator  O.  Figure  5.2  shows  the  generation  of  a 
new  image  point  p0  in  level  fc  -  1  from  the  quadrupel  of  the  image  points  in  the 
level  k  by  using  O.  This  mapping  is  defined  by  the  operator  O.  such  that: 


«(*-')  - 


-=  S(k>  SW  3 

i  ’  Spi  ’  sp,  ■  3 ; 


Appropriate  operators  O.  for  example,  are: 


4 


Oavg  - 

t  =  l 

4 

^min  = 

min 

1=1 

C^max  = 

max 

i=i  p* 

The  generation  of  multiresolution  gray  level  pyramids  can  be  done  by  any 
smoothing  algorithm.  The  structural  levels  of  the  pyramid  show  a  growing  dis¬ 
appearance  of  the  finer  details  in  the  levels  of  worse  resolution.  The  gray  level 
pyramid  of  structural  levels  8  to  0  resulting  from  applying  an  unweighted  aver¬ 
aging  operator  to  the  original  image  is  shown  in  Figure  5.3. 
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Figure  5.2:  Generation  of  a  pyramidal  image  point  p0  at  level  k  -  1 
from  the  quadrupel  (pt ,  p7,  P3,  p<)  at  level  k 
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5.1.2  Projection  of  an  Image  Point 

For  further  steps  of  the  image  preprocessing  it  is  of  interest  to  project  an  image 
point  s\kj  of  level  k  into  the  level  k  +  i,  with  i  €  {0,1,  (N  -  k)}.  When 
projecting  such  a  point  from  level  k  into  level  k  4-  t,  another  pyramid  is  formed. 
The  quadratic  base  of  it  is  called  macro  pixel  M.  The  size  2*  2*  of  the  macro 
pixel  M  depends  on  the  difference  between  the  spot  level  k,  where  the  p'mt  to 
be  projected  is  located,  and  the  projection  level  k  +  l. 


5.2  Contrast  Pyramids 

Features  are  locally  extracted  from  each  image  point  of  the  gray  level  pyramid 
by  gradient  and  curvature  filtering.  This  is  done  by  means  of  difference  and 
Laplacian  like  operators.  Further  image  analysis  is  based  on  the  resulting  fil¬ 
tered  images. 

The  gradient  of  an  image  function  s  is  given  by 

(  s{x  +  Ax, y)  -  s(x,y)  \ 

lim  - r - 

il-0  Ax 

s(x,y  -e  Ay)  -  s(x,y) 

hm - - - - 

V  a»-.o  A y  / 


grad  s(x,y)  = 


/  ds(x>y>  \ 

dx 

ds(x,y) 


dv 


the  direction  and  amount  of  which  are: 


os(x,y) 

V5(grads(x,t/))  =  arctan--^— 

dx 


j  ,  v  \(ds{x,y)\  (ds(x,y)\‘ 

Igrad  s(x,y)|  = 


The  Laplacian  of  an  image  function  s  is  given  by 


V  s(x 


,y)  =  ^ 


d2s(x,y)  d2s(x.y) 


dx1 


dy1 


In  the  following  two  sections  two  simple  operators  are  introduced  approximat 
ing  gradient  and  curvature  filtering  from  the  gray  value  pyramid. 


5.2.1  Detection  of  Edge  Elements 

In  order  to  detect  edge  elements  from  local  differences  of  intensity  in  the  image 
function  s,  the  Sobel  operator  G  =  (GI,Gy)  is  used:  , 
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The  image  function  s(x,y)  is  transformed  into  the  functions  gx(x,y)  and 
gv(x,y)  by  convolving  s  with  the  masks  Gx,  resp.  Gy  of  the  Sobel  operator: 

9x{x,y)  =  (a(z  +  1,2/  -  1)  +  2s(x  +  l,y)  +  s(x  +  l,y  +  1))  - 

(s(z  -  1  ,y  -  1)  +  2s(x  -  l,y)  +  s(x  -  l,y  +  1)) 

9v{x,y)  =  (s(x  -  l,t/  +  1)  +  2s(x,y  +  1)  +  s(x  +  l,y  +  1))  - 

(s(i  -  l,y  -  1)  +  2 a(x,y  -  1)  +  s(x  +  l,y  -  1)) 

Then  the  absolute  values  g(x,y )  and  the  directions  y>(x,y)  of  the  gradients  are 
calculated  from  gx  and  gy: 

9{x,y)  =  \]9x{x,y)2  +  9v(x,y)2 

,  s  .  9„{x,y) 

ip{x,y)  =  arctan — - - - 

9*{x,y) 

The  directions  of  the  gradient  {0,  27r}  represent  the  vectorized  orien¬ 

tations  of  an  edge  element  after  a  positive  rotation  of  7r/2  of  gradients.  The 
orientations  of  an  edge  element  can  also  be  transformed  into  {0,  . . .  ,  rr}  with 
included  notation  of  its  positive  or  negative  contrast  step.  The  gradients  are 
transformed  into  vectorized  orientations  of  an  edge  element  and  quantized  into 
the  discrete  values  {0,  ...,  7}  representing  the  directions  of  the  7V8  neighbor¬ 
hood  (see  Figure  5.8a).  After  calculation  of  g  and  ip  their  values  are  entered 
into  the  absolute  value  pyramid  or  direction  pyramid,  respectively.  Both  pyra¬ 
mids  are  used  for  the  extraction  procedure  of  edge  segments  (see  section  5.3). 

The  absolute  value  and  the  direction  of  the  gradients  calculated  from  the 
Sobel  filtered  contrast  image  PHOENIX-2A1  in  best  resolution  (structural  level 
S)  are  shown  in  Figure  5.4  and  Figure  5.5.  In  order  to  display  the  results,  the 
gray  values  are  enhanced  to  (0,  ...,  255}.  The  other  levels  8  to  0  of  the  abso¬ 
lute  value  and  direction  pyramid  show  similar  results  and  are  not  given  here. 

Finally  it  is  to  be  mentioned  that  instead  of  the  Sobel  operator  also  other 
edge  detecting  operators  could  be  used. 


5.2.2  Detection  of  Spots 


In  order  to  detect  spots  from  local  differences  of  intensity  in  the  image  function 
s,  the  spot  operator  U  for  curvature  is  used- 


U  = 


/  -1  -1  -1 
-1  8  -1 
V  -1  -I  -1 
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Figure  5.4:  Absolute  value  of  gradients  of  PHOENIX -2A1 
in  best  resolution  (level  9) 

(absolute  values  {0,  . . . ,  gmal}  are  enhanced  to  the  gray  values  {0,  . . . ,  255}) 
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Figure  5.5:  Direction  of  gradients  of  PH0ENIX-2A1  in  best  resolution  (level  9) 
(directions  {0,  . . .  ,  2ir}  are  transformed  to  the  gray  values  {0,  . . . ,  255}) 
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The  image  function  3(2, y)  is  transformed  into  the  function  u (x,y)  by  con¬ 
volving  s  with  the  mask  U  of  the  spot  operator: 

u(*,y)  =  ~3(x-l,y-l)  -3(2 -l,y)  —s(i  —  l,y  4-  1) 
-3(z,y-l)  +83(2, y)  -3(2, y  4-1) 

-3(1  4- l,y  -  1)  —3(1  4- 1 , y)  -j(i4l,y+l) 

In  order  to  receive  shape  features  from  local  differences  of  intensity  in  the 
image  function  3,  the  operator  V  =  (14,  V),  V2,  V3,  V4,  V6,  V*,  Vr)  is  used  beside 
U: 


1  1 

-K 

i 

kT  '-©c 

1  1 

with 

6,  =  { 

\ 

-K 

-K  y 

for  j  —  i 
for  j  /  t 


The  image  function  3(2,  y)  is  transformed  into  the  functions  ^(2,?/)  by  con¬ 
volving  3  with  the  masks  V;  of  the  shape  feature  operator  V: 


^(*,y)  =  ~K,3(x  ~  i-y  - 1)  ~st,3(x  -  i,y)  -^,3(2  -  i,y  4- 1) 
-^3(2,y  -  1)  +3(2,  y)  —6„)s(x,  y  4-  1) 

-$»>«(*  4-  l,y  -  1)  -62js(x  +  l,y)  -6us(x  4-  l,y  +  1) 

A  useful  shape  measurement  is  the  socalled  surroundedness  v(x ,y)  featuring 
spots.  This  feature  denotes  the  number  of  image  elements  in  the  NB  neighbor¬ 
hood  of  a  spot  which  are  brighter  or  darker  than  the  dark,  resp.  bright  con¬ 
trasting  spot: 


v(*,V)  =  2^  sgnMx>2/)(s6n“(*,y)  +  sgni;,(2,y)) 

The  next  two  figures  show  the  results  from  filtering  the  images  of  the  gray 
value  pyramid  (structural  levels  8  to  0)  with  both  the  spot  detection  and  spot 
feature  operator.  Figure  5.6  presents  the  results  from  spot  detection.  The 
brighter  the  displayed  values  from  bright  spots  and  the  darker  from  dark  spots 
are,  the  greater  are  the  features  curvature  imaged  to  the  values  128  to  255  or 
127  to  0,  respectively,  wherby  the  value  between  127  and  128  represents  zero 
curvature.  Figure  5.7  shows  the  surroundedness  of  spots  from  the  Ne  neigh¬ 
borhood.  The  brighter  the  displayed  values  from  the  bright  and  dark  spots 
are,  the  greater  are  the  features  surroundedneaa  imaged  to  the  values  0  to  255. 
Both  pyramids  are  used  for  the  extraction  procedure  of  area  like  segments  (see 
section  5.4). 

5.3  Edge  Oriented  Segmentation 

The  edge  oriented  segmentation  procedure  is  based  on  edge  tracking.  The 
tracking  is  performed  by  gradient  controlled  following  of  edge  elements.  Start¬ 
ing  points  for  edge  tracking  are  such  image  points  in  the  pyramid  where  a  high 
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Figure  5.6:  Spot  curvature  pyramid  of  PH0ENIX-2A1  (levels  8  to  0) 
(curvatures  {-|u|m„  . . . ,  0,  . . . ,  |u|m„}  are  enhanced  to 
the  gray  values  {0,  . . . ,  255}) 
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Figure  5.7:  Spot  surroundedness  pyramid  of  FHOHNIX.2A1  (levels  8  to  0) 
(features  {0,  .  . . ,  8}  for  surroundedness  are  enhanced  to 
the  gray  values  {0,  . . . ,  255}) 
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absolute  value  of  gradients  are  found  after  having  used  the  edge  operator.  Con¬ 
trary  to  other  edge  tracking  procedures  described  in  the  literature  (see,  for  ex¬ 
ample,  [1,7,13]),  it  is  not  aimed  at  processing  the  gradient  image  area  totally 
or  in  parallel.  Instead  of  that  it  is  tried  to  get  connected  edges  sequentially  by 
following  the  orientation  of  edge  elements  starting  with  one  of  the  most  reli¬ 
able  edge  elements.  The  edge  segmentation  algorithm  is  based  on  the  following 
steps  explained  in  detail  in  the  next  subsections: 

1.  preparation  of  the  starting  point  list  for  edge  segmentation 

2.  extraction  of  edge  segments  and  features  by  tracking  of  edge  elements 

3.  assessment  ol  edge  elements  and  segments 

4.  approximation  of  edge  segments  by  polygons 

5.  generation  of  symbolic  description  of  edge  segments 

5.3.1  Starting  Point  List  for  Edge  Segmentation 

The  coordinates  of  pixels  representing  edge  elements,  which  in  each  resolution 
level  are  sorted  in  descending  order  ^responding  to  the  magnitude  of  absolute 
value  of  their  gradients,  are  compiled  in  a  starting  point  list.  This  list  then  is 
read  from  that  end  with  the  highest  absolute  value.  Thereby  it  is  guaranteed 
that  the  edge  extraction  always  begins  with  the  most  reliable  edge  element. 
The  procedure  stops  when  a  given  threshold  of  the  magnitude  of  absolute  value 
is  reached. 

5.3.2  Tracking  of  Edge  Elements  and  Feature  Extraction 

The  edge  tracking  starts  with  the  track  parameter  t  =  0  at  the  actual  starting 
point  Pot^oil/o)  contained  in  the  starting  point  list  S.  After  tracking  of  edge 
elements  p0(xt,yt)  the  segment  is  described  by  a  string  x,,yt  of  coordinates  of 
the  actual  pyramidal  level.  Possible  candidates  p;  to  be  added  to  each  actual 
pixel  pa  of  an  edge  element  at  the  position  xt,p(  are  contained  in  a  subset  of 
pixels  in  the  IV8  neighborhood  of  p0.  The  pixel  p,  can  be  addressed  by  the 
direction  Figure  5.8  shows  the  relation  between  the  coding  of  the  directions 
'll  and  the  JV8  neighborhood.  That  scheme  is  also  used  for  the  coding  of  the 
gradient  of  an  actual  point  po  as  edge  direction  <J>. 

The  pixels  p,  connected  to  the  actual  pixel  po  for  edge  segmentation  are  dis¬ 
tinguished  by  successors  pf  and  predecessors  pf.  This  means  that  it  is  tried  to 
track  the  edge  to  both  ends  with  t  >  0  for  (5)  and  t  <  0  for  (P)  beginning  with 
the  starting  point  t  =  0.  Connecting  N~  preceding  and  N+  succeeding  pixels 
the  string  (x/y- i  P/v- >  xo,  Pol  ii.Pii  •••;  xn*  >  Vn*  )  of  coordinates  of 

edge  elements  is  generated.  For  example,  Figure  5.9  shows  the  selected  candi¬ 
dates  for  edge  tracking  related  to  an  actual  point  p0  with  an  arbitrary  gradient 
coded  as  edge  direction  =  6  (symbolized  by  T)- 
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Figure  5.8:  Coding  scheme  a)  of  directions  $  or  f  for  b)  edges,  resp.  c)  neigh¬ 
bors 


Figure  5.9:  Edge  tracking  candidates  for  a)  successors,  resp.  b)  predecessors 
with  respect  to  an  actual  point  p0  with  a  gradient  coded  as  edge  direction 


$  =  6 
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The  successors,  resp.  predecessors  are  addressed  by  the  direction  't  derived 
from  the  edge  direction  $  of  the  actual  point  p0.  In  order  to  select  the  succeed¬ 
ing  ps,  resp.  the  preceding  pF  candidates  for  a  point  p0,  the  following  rules  to 
compute  the  direction  W  in  p0  from  $  are  used,  where  i  =  1  is  the  left,  i  =  2 
the  central,  and  t  =  3  the  right  successor,  resp.  vice  versa  for  the  predecessor: 


[  ($(p0)-l) 

mod  8 

for  i  =  1 

*(pf)  =  • 

,  $(?o) 

for  i  =  2 

[  ($(po)  +  l) 

mod  8 

for  i  =  3 

f  (*(Po)-5) 

mod  8 

for  i  =  1 

*(pf)  =  - 

o 

1 

mod  8 

for  i  —  2 

[  (*(pb)-3) 

mod  8 

for  i  =  3 

The  surrounding  pixels  p; 

of  the  actual  edge  element  p0  are  examined  in 

order  to  choose  the  best  successor,  resp.  the  best  predecessor  among  the  can¬ 
didates  by  means  of  criteria  to  be  defined  in  the  next  subsection.  Taking  the 
starting  point  p0,  it  is  tried  to  track  the  edge  elements  in  both  directions  un¬ 
til  both  end  points  xn+,Vn*  and  xs-,yN -  are  reached.  Considering  a  tracking 
in  the  direction  of  successors  those  end  points  represent  the  starting  point  (for 
N~)  and  the  end  point  (for  N+)  of  edge  segments.  After  tracking  of  edge  ele¬ 
ments  radiometric  and  geometric  features  of  edge  segments  are  generated. 

5.3.3  Assessment  of  Edge  Elements  and  Segments 

Assessment  of  edge  segmentation  is  done  by  two  steps.  First,  edge  elements 
which  are  candidates  for  edge  tracking  are  selected  from  the  Ns  neighborhood 
at  both  ends  of  a  segment  generated  so  far.  These  elements  have  to  meet  some 
minimum  requirements  for  edge  tracking,  and,  if  these  qualifications  are  not 
satified,  the  edge  segment  with  the  element  concerned  is  not  continued  at  this 
end.  Each  edge  element  which  can  be  considered  as  a  candidate  for  edge  track¬ 
ing  is  assessed  with  respect  to  direction  and  absolute  value  of  the  gradient  for 
compatibility  with  those  values  of  the  already  existing  edge  element  belonging 
to  the  segment.  After  that  a  total  assessment  of  all  candidates  is  made,  and 
the  candidate  with  the  highest  value  is  taken  to  be  added  to  the  edge  segment 
as  next  edge  element. 

Second,  if  the  edge  segment  cannot  be  continued  at  both  ends  furthermore, 
a  final  assessment  of  the  edge  segment  is  done.  If  this  final  assessment  does  not 
meet  the  minimum  requirements  for  a  segment,  the  complete  edge  segment  is 
omitted. 

Compatibility  of  Directions 

Testing  the  compatibility  of  directions  of  edge  elements  means  to  compare  the 
gradient  direction,  resp.  edge  direction  of  a  candidate  pixel  pi  with  that  of  the 
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Figure  5.10:  Ranking  of  compatible  successors  with  respect  to  edge  directions 
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Figure  5.11:  Ranking  of  compatible  predecessors  with  respect  to  edge  direc¬ 
tions 


actual  point  po  of  an  edge  segment  already  existing.  Which  directions  between 
neighboring  candidates  and  actual  edge  element  are  compatible  and  how  they 
are  assessed,  is  shown  exemplarily  in  Figures  5.10  and  5.11  for  an  edge  element 
with  an  arbitrary  edge  direction  6  (symbolized  by  t).  If,  for  example,  the  can¬ 
didates  pi  and  p3  have  the  edge  direction  5  (symbolized  by  \),  then  pi  gets  a 
higher  assessment  than  P3.  If  the  difference  of  directions  between  a  neighbor¬ 
ing  candidate  and  the  actual  edge  element  is  greater  than  2,  then  the  candidate 
gets  the  assessment  0  and  is  not  added  to  the  edge  segment. 


Compatibility  of  Absolute  Values 

Concerning  the  absolute  values  of  gradients  the  compatibility  of  a  candidate 
pixel  p,  depends  on  the  difference  between  its  own  value  and  the  values  of  that 
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points  of  the  edge  segment  already  existing.  In  order  to  examine  this  compati¬ 
bility,  a  linear  regression  is  made  with  the  absolut*  values  of  the  gradients  be¬ 
longing  to  the  edge  segment  existing  so  far.  An  extrapolation  by  means  of  the 
computed  regression  line  gives  a  prediction  g1  for  the  absolute  gradient  value  of 
the  candidate  to  be  tested.  The  smaller  the  difference  between  the  values  g  of 
the  candidate  and  the  predicted  value  g',  the  higher  is  the  assessment.  If  its 
amount  is  higher  than  a  given  threshold  the  candidate  is  assessed  to  0  and  not 
added  to  the  edge  segment. 

Total  Compatibility 

The  total  assessment  for  each  of  the  candidate  is  a  combination  of  both  assess¬ 
ments  for  direction  and  absolute  value  of  gradients.  The  candidate  with  the 
highest  assessment  is  added  to  the  edge  segment.  If  the  assessment  of  all  can¬ 
didates  is  zero  no  pixel  from  the  Ng  neighborhood  is  connected  to  the  actual 
point  and  the  edge  segment  is  not  continued  at  this  end. 

Final  Compatibility 

In  case  that  the  continuation  of  an  edge  is  no  more  possible,  the  total  length  of 
the  complete  edge  segment  and  its  mean  absolute  value  from  linear  regression 
is  assessed.  The  entire  edge  segment  is  rejected,  if  a  given  minimum  of  mean 
absolute  value  and  a  minimum  length  are  not  reached.  Figure  5.12  shows  the 
result  of  edge  oriented  segmentation  by  tracking  of  edge  elements.  Different 
edges  are  displayed  by  differing  colors.  Only  the  best  resolution  level  9  of  the 
edge  pyramid  is  shown.  The  results  from  the  other  resolution  levels  8  to  0  are 
comparable  to  that  of  level  9. 

5.3.4  Approximation  of  Edge  Segments  by  Polygons 

After  the  edge  segments  have  been  found,  they  are  analyzed  with  respect 
to  global  segment  features  and  are  approximated  by  polygons  the  vertices  of 
which  are  only  stored.  Then  the  original  strings  of  pixel  coordinates  from  the 
edge  elements  of  the  segments  are  cancelled.  Each  detected  edge  segment  is 
described  by  the  edge  parts  between  its  vertices.  The  edge  parts  are  received 
by  connecting  selected  pixels  (sample  points)  of  the  original  edge  segment  with 
straight  lines.  The  sample  points  of  the  r'.i  inal  edge  segment  represent  the 
vertices  of  the  approximated  edge  segmem  ■'  no  .  "on).  The  first  two  vertices  are 
the  first  and  the  last  point  of  the  original  e<'"  egment  (end  points).  The  next 
vertex  is  that  point  lying  on  the  original  edge  segment  which  has  the  largest 
distance  d  to  the  line  drawn  through  the  starting  point  and  the  end  point.  This 
vertex  divides  the  original  edge  segment  into  two  edge  parts,  ^hen  a  new  ver- 
tix  is  determined  for  each  edge  part  until  the  maximum  distance  d  of  each  edge 
part  to  the  original  edge  segment  is  smaller  than  a  given  threshold. 
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Figure  5.12:  Edge  oriented  segmentation  of  PH0ENIX.2A1  in  best  resolution 
(level  9)  (different  edges  are  displayed  by  differing  colors) 
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5.3.5  Symbolic  Description  of  Edge  Segments 

The  iconic  results  of  edge  segmentation  are  transferred  to  symbolic  descriptions 
for  further  analyzation.  In  order  to  present  edge  segments  symbolically,  fea¬ 
tures  (symbols)  are  generated  from  each  original  and  approximated  edge  seg¬ 
ment  their  attributes  of  which  are  inserted  into  two  lists.  Three  different  types 
of  attributes  are  allocated  to  both  lists: 

1.  key  data 

2.  radiometric  features 

3.  geometric  features 

In  order  to  link  global  information  of  segments  and  details  of  edge  parts 
additional  attributes  are  required  for  the  first  list: 

4.  directory  data 

Figure  5.13  shows  the  data  structure  of  both  lists.  The  first  list 
(symbol0-list)  contains  the  global  features  of  a  nonapproximated  edge  segment 
and  a  pointer  to  the  second  list  (symboll .  list),  in  which  the  corresponding 
data  for  the  edge  parts  after  edge  approximation  are  contained.  The  term  con¬ 
trast  used  in  the  lists  means  the  absolute  value  of  the  gradient  and  the  term 
slope  refers  to  the  linear  regression  applied  to  the  contrast. 


5.4  Area  Oriented  Segmentation 

The  procedure  of  the  area  oriented  segmentation  can  be  characterized  as  a  lo¬ 
cal  binarization  method.  Homogeneous,  contrasting  areas  of  different  size  in 
a  gray  level  image  are  presented  in  appropriate  pyramidal  resolution  levels  as 
bright,  resp.  dark  contrasting  spots.  Similar  to  edge  tracking  the  sequential 
procedure  for  area  oriented  segmentation  is  driven  by  the  best  assessed  spots. 
The  segmentation  method  produces  the  contours  of  bright,  resp.  dark  contrast¬ 
ing  areas.  The  segmentation  algorithm  for  areas  is  based  on  the  following  steps 
explained  in  detail  in  the  next  subsections: 

1.  preparation  of  the  starting  point  list  for  area  segmentation 

2.  projection  of  spots  into  levels  of  higher  resolution 

3.  generation  of  area  segments  by  binarization 

4.  extraction  of  area  segments  and  features  by  tracking  of  contour  elements 

5.  assessment  of  area  segments 

6.  approximation  of  area  segments  by  polygons 

7.  generation  of  symbolic  descriptions  of  area  segments 
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5.4.1  Starting  Point  List  for  Area  Segments 

For  each  resolution  level  of  the  spot  pyramid  the  spots  are  sorted  by  two  steps 
in  descending  order  with  respect  to  to  their  values  of  curvature  and  surround- 
edness.  Their  coordinates  are  compiled  in  a  starting  point  list.  The  most  re¬ 
liable  spot  is  that  with  the  greatest  negative  value  (for  dark  spots),  resp.  the 
greatest  positive  value  (for  bright  spots)  of  curvature,  and  with  the  surround- 
edness  of  8.  The  spot  operator  for  curvature  introduced  in  subsection  5.2.2  pro¬ 
vides  values  lying  between  —2040  (dark  spots)  and  +2040  (bright  spots).  These 
values  u  are  normalized  to  u'  so  that  0  <  u'  <  255,  i.e.  the  best  assessment  for 
a  dark  spot  is  0,  and  the  best  assessment  for  a  bright  spot  is  255. 

For  each  resolution  level  first  the  spots  are  sorted  in  descending  order  ac¬ 
cording  to  their  surroundedness.  Second,  the  spots  are  sorted  in  descending 
order  for  each  class  of  surroundedness  according  to  their  curvature.  Then  the 
starting  point  list  is  read  from  the  bottom  (dark  spots  to  segment  dark  ar¬ 
eas),  resp.  from  the  top  (bright  spots  to  segment  bright  areas)  until  a  certain 
threshold  for  curvature  and  surroundedness  is  reached.  Thus  it  is  guaranteed 
that  only  the  most  reliable  spots  are  selected  for  area  segmentation.  Regarding 
thresholds  experience  has  shown  that  values  for  the  surroundedness  should  lie 
between  8  and  4,  and  the  values  for  the  curvature  below  126  for  dark  spots  and 
above  129  for  bright  spots. 


5.4.2  Spot  Projection  into  Levels  of  Higher  Resolution 

The  spots  in  the  resolution  level  k  of  the  spot  pyramid  are  cues  for  the  de¬ 
tection  of  homogeneous  areas  in  the  levels  k  +  l  of  higher  resolution.  Further¬ 
more  the  pyramidal  structure  gives  information  about  the  possible  size  of  the 
expected  areas  depending  on  the  difference  t  between  the  level,  in  which  the 
spot  is  detected  (spot  level  SL),  and  that  level  into  which  the  spot  is  projected 
(projection  level  PL).  Figure  5.14  shows  schematically  the  projection  of  spots 
from  the  spot  level  into  the  projection  level. 

A  spot  at  the  resolution  level  k  determines  an  area  with  the  size  of  2‘  x  2* 
pixels  at  the  resolution  level  k  +  i.  This  area  is  the  socalled  macro  pixel  M 
the  length  of  which  amounts  to  2‘.  The  border  coordinates  m(M),n(M)  of  the 
macro  pixel  M  depend  on  the  coordinates  m,n  of  the  spot  at  level  k  and  on 
the  difference  i  between  projection  and  spot  level: 


m.tar  t(M)  -  (m  —  1)  2‘  +  1 
nitart(M)  —  (n  —  1)2‘  +  1 


=  m  2‘ 
=  n  2‘ 


and 


m€nd(M) 

Tlend{  M  ) 
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Figure  5.14:  Projection  of  spots  into  levels  of  higher  resolution 
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The  segmentation  area  is  defined  by  the  socalled  patch  P,  the  size  of  which 
is  3  x  3  macro  pixels.  The  edge  length  of  a  patch  amounts  to  3  •  2‘.  The  border 
coordinates  m(P),n(P)  of  the  patch  P  depend  on  the  coordinates  m,n  of  the 
spot  at  level  k  and  on  the  difference  i  between  projection  and  spot  level,  thus 
on  the  coordinates  m(M),n{M)  of  the  macro  pixel  M: 


m,tart(P )  =  2m,,art{M)  -  mend(M)  -  1 

tart(P)  —  2nJ(art(A/)  —  ncnd(M)  —  1 


and  mend(P)  =  2m,nJ(M)-mlta„(M)  +  l 
nertd(P)  =  2  nend(M)  -  n,tar,(M)  +  1 

The  minimum  and  the  maximum  size  of  the  area  segment  to  be  detected  is 
determined  by  the  difference  i  of  projection  level  and  spot  level.  The  approxi¬ 
mate  minimum  size  is  as  large  as  the  macro  pixel,  whereas  the  maximum  size 
can  be  as  large  as  the  patch  in  which  the  segmentation  is  performed. 


5.4.3  Area  Segmentation  by  Binarization 

The  choice  of  the  binarization  threshold  within  the  patch  is  of  decisive  im¬ 
portance  for  the  segmentation  result.  This  local  threshold  is  seperately  calcu 
fated  for  each  selected  spot  candidate,  and  is  applied  to  each  structural  level  of 
the  gray  value  pyramid  with  a  higher  resolution  than  the  spot  level  (see  Fig¬ 
ure  5.14).  The  binarization  threshold  depends  on  the  gray  values  of  the  macro 
pixel  in  the  projection  level  of  the  gray  level  pyramid  concerned  (see  section 
4.1.1).  The  binarization  thresholds  0;,  for  bright  and  Qd  for  dark  spots  are: 

Oi,  =  s  -  f  <r, 

=  s  +  { <T, 

Thereby  s  is  the  unweighted  medium  gray  value  of  the  macro  pixel  and  <r,  is 
the  standard  deviation  of  the  gray  values.  The  factor  /  allows  to  raise  or  to 
lower  the  effect  of  area  binarization  .  An  increase  of  the  factor  leads  to  a  de¬ 
crease,  resp.  increase  of  the  binarization  threshold  for  bright,  resp.  dark  spots, 
which  leads  to  the  detection  of  more  and  larger  area  segments.  Concerning 
bright,  resp.  dark  spots  having  gray  values  being  greater,  resp.  less  than  the 
threshold,  then  connecting  pixels  are  merged  to  a  bright,  resp.  dark  area.  The 
connecting  pixels  are  labelled  by  1  or  any  other  positive  numeral,  the  back¬ 
ground  by  0. 
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5.4.4  Tracking  of  Contour  Elements  and  Feature  Ex¬ 
traction 


In  order  to  determine  the  contour  of  an  area  segment,  a  contour  tracking  algo¬ 
rithm  given  in  [14]  is  used.  The  aim  is  to  encircle  the  connected  labelled  points 
of  the  segment  first  and  then  to  fill  possible  gaps  within  the  area.  The  contour 
tracking  algorithm  can  be  described  as  follows:  The  upper  left  1-labelled  bor¬ 
der  point  within  the  patch  is  chosen  as  a  starting  point  5.  Starting  with  this 
point  the  contour  points  are  surrounded  by  a  2  x  2  mask  and  new  labelled  de¬ 
pending  on  the  change  of  the  direction  of  the  inner,  resp.  outer  contour  points. 
The  elements  of  the  mask  are  denoted  by  P,  Q,  V,  U  with  an  arrow  between  P 
and  Q  pointing  to  the  centre  of  the  mask. 

At  the  beginning  of  the  contour  tracking  the  element  P  of  the  mask  is  fixed 
to  the  starting  point  S  of  the  segment  and  Q,V,  U  lie  outside  the  area  segment. 
Then  the  mask  is  rotated  and  shifted  such  that  always  at  least  P  of  the  four 
elements  of  the  mask  lies  on  the  inner  contour  of  the  area  segment.  The  mask 
is  rotated  around  P  or  Q,  and  shifted  in  the  direction  of  the  arrow  substituting 
the  old  V,  U  by  the  new  Q ,  P.  The  new  orientation  or  position  of  the  mask  is 
always  determined  by  substituting  only  the  two  elements  P  and  Q.  Figure  5.15 
shows  the  tranfer  table 


( P,Q,V,U)t 


1{U),,L(Y), 


depending  on  the  labels  {1,0}  of  area  points  of  the  corresponding  mask  ele¬ 
ments  U,  V  denoted  by  L(U),  L(V).  The  contour  tracking  has  finished  if  the 
starting  point  5  is  reached  again.  Between  the  marked  points  the  area  segment 
is  filled  up  with  the  given  1-label.  Thereby  0-distortions  resulting  from  the  bi- 
narization  procedure  are  eliminated.  After  tracking  of  the  contour  elements  ra¬ 
diometric  and  geometric  features  of  the  area  segment  are  generated. 


5.4.5  Assessment  of  Area  Segments 

The  area  segments  are  assessed  with  respect  to  geometric  and  radiometric  fea¬ 
tures.  These  features  serve  as  thresholds  in  order  to  decide  if  a  segment  is  to 
be  accepted  or  has  to  be  rejected.  The  following  features  are  used  for  thresh¬ 
olding: 

•  area  size 

•  compactness 

•  absolute  values  of  contour  gradients 

•  homogeneity  of  gray  vab.es  within  the  area 

As  segmentation  occur-,  only  within  the  patch,  the  magnitude  of  area  sizes 
to  be  obtained  depends  on  the  difference  between  projection  and  spot  level.  A 
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t  =  4 


Figure  5.15:  Contour  tracking  starting  from  point  S  ((14]) 
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segment  is  then  accepted,  if  its  area  A  lies  between  f\M  <  A  <  /j  M.  Thereby 
M  is  the  size  of  the  macro  pixel,  /;  and  /2  are  factors  with  0.0  <  f\  <  1.0 
and  1.0  <  fj  <  9.0.  In  order  to  avoid  finding  too  fuzzy  segments,  only  seg¬ 
ments  above  a  minimum  compactness  are  allowed.  Segments  with  a  mean  ab¬ 
solute  value  of  contour  gradients  below  a  given  threshold  are  rejected  as  well  as 
segments  with  a  standard  deviation  of  gra.  values  being  greater  than  a  given 
threshold  for  low  homogeneity. 

A  confidence  measure  for  each  accepted  segment  is  calculated  by  the 
weighted  sum  of  single  confidence  values.  Those  single  values  are  the  area  size 
A,  the  compactness  C,  the  mean  absolute  value  of  contour  gradients  K,  and 
the  standard  deviation  of  the  gray  values  E  of  the  area  segment,  which  are  pro¬ 
vided  with  eligible  weighting  factors  gi  with  the  sum  of  them  is  equal  to  1: 

V  =  g\A  +  g2C  +  g$K  +  g,|£  with  fli  +  gz  +  fl3  +  9<  —  1 

5.4.6  Approximation  of  Area  Segments  by  Polygons 

The  tracked  contour  lines  of  area  segments  are  temporarily  recorded  as  lists 
of  coordinates  for  feature  extraction.  In  order  to  reduce  storage  and  comput¬ 
ing  time  the  area  segments  are  approximated  by  closed  polygons  the  vertices 
of  which  are  stored  permanently.  The  procedure  for  approximating  area  seg¬ 
ments  works  similar  to  that  for  edge  segments  described  in  section  5.3.4.  The 
detected  contour  is  described  by  its  contour  parts.  These  parts  are  found  by 
connecting  sample  points  of  the  original  contour  with  straight  lines,  the  sam¬ 
ple  points  becoming  tne  vertices  of  the  approximating  polygon.  The  first  both 
sample  points  are  an  arbitrarily  selected  contour  point  Plt  e.g.  the  starting 
point  5,  and  that  point  P2  or  the  contour  with  the  maximum  distance  to  Pi, 
resp.  5.  Both  points  are  connected  by  a  line  which  divides  the  original  contour 
into  two  parts.  The  next  two  vertices  are  those  two  points  of  the  contour  which 
are  most  far  away  from  this  line.  By  this  means  the  original  contour  line  is  re¬ 
placed  by  four  contour  parts.  This  procedure  is  continued  until  the  maximum 
distance  of  a  line  to  the  contour  is  smaller  than  a  given  threshold. 

Segmentation  results  for  two  different  local  binarization  thresholds  with  fac¬ 
tors  /  chosen  as  /  =  —0.2,  resp.  /  =  +0.2  are  shown  in  Figures  5.16  and  5.17 
(bright  areas),  resp.  in  Figures  5.18  and  5.19  (dark  areas).  The  bright  and  dark 
areas  are  segmented  from  the  best  resolution  level  9  of  the  pyramid  with  the 
spots  as  cues  projected  from  level  6  to  level  9.  The  different  areas  are  displayed 
by  different  colors  in  order  to  distinguish  the  detected  segments.  The  horizon¬ 
tal  and  vertical  contour  lines  are  due  to  the  borders  of  the  patches. 

When  regarding  other  spot  and  projection  levels,  smaller  or  larger  area  seg¬ 
ments  would  be  extracted  by  this  method.  For  example,  segmentation  results 
for  one  local  binarization  threshold  with  factor  /  chosen  as  /  =  —0.2  are  shown 
in  Figure  5.20  (bright  areas)  and  in  Figure  5.21  (dark  areas).  The  bright  and 
dark  areas  are  segmented  from  the  best  resolution  level  9  of  the  pyramid  with 
the  spots  as  cues  projected  from  level  5  to  level  9. 
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Figure  5.16:  Area  oriented  segmentation  of  PHOENIX-2A1 
in  best  resolution  (level  9) 

(bright  areas,  binarization  factor  /  =  -0.2,  projection  from  level  6  to  level  9) 
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Figure  5.17:  Area  oriented  segmentation  of  PHOENIX.2A1 
in  best  resolution  (level  9) 

(bright  areas,  binarization  factor  /  =  +0.2,  projection  from  level  6  to  level  9) 


CHAPTER  5.  ICONIC  IMAGE  FILTERING  AND  SEGMENTATION  103 


Figure  5.18:  Area  oriented  segmentation  of  PHOENIX.2A1 
in  best  resolution  (level  9) 

(dark  areas,  binarization  factor  /  =  -0.2,  projection  from  level  6  to  level  9) 


CHAPTER  5.  ICONIC  IMAGE  FILTERING  AND  SEGMENTATION  104 


a 


r  l  —'.  •  :  '  '  o 

<1  f  '  >?/f 


*i 


•5^rn‘: 


-i  i',.f 


•|| 


F 


I  , 


W  i; 


"Tlfi  a'l-%. 


I  . ' 


^  r  I- 

'  r  ;f 


■T  i  ■b-’-c 

;  <^n  ? .:  r 


-  ► 


'  ■  r_j  * 

[  .•'!  ii  r  n  *_ 


.. 1  * 

>  f' 

>L 


-  »j- ' 


M 


Figure  5.19:  Area  oriented  segmentation  of  PH0ENIX-2A1 
in  best  resolution  (level  9) 

(dark  areas,  binarization  factor  /  =  +0.2,  projection  from  level  6  to  level  9) 
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Figure  5.20:  Area  oriented  segmentation  of  PHOENIX-2A1 
in  best  resolution  (level  9) 

(bright  areas,  binarization  factor  /  =  —0.2,  projection  from  level  5  to  level  9) 
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Figure  5.21:  Area  oriented  segmentation  of  PHOENIX-2A1 
in  best  resolution  (level  9) 

(dark  areas,  binarization  factor  /  =  -0.2,  projection  from  level  5  to  level  9) 
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5.4.7  Symbolic  Description  of  Area  Segments 

The  iconic  results  of  area  segmentation  are  transferred  to  symbolic  descriptions 
for  further  analyzation.  In  order  to  present  area  segments  symbolically,  fea¬ 
tures  (symbols)  are  generated  from  each  original  and  approximated  area  seg¬ 
ment  their  attributes  of  which  are  inserted  into  two  lists.  Analog  to  the  treat¬ 
ment  of  edge  segments  (see  section  5.3.5)  three  different  types  of  attributes  are 
allocated  to  both  lists: 

1.  key  data 

2.  radiometric  features 

3.  geometric  features 

In  order  to  link  global  information  of  segments  and  details  of  contour  parts 
additional  attributes  are  required  for  the  first  list: 

4.  directory  data 

Figure  5.22  shows  the  data  structure  of  both  lists.  The  first  list 
(symbol0_list)  contains  the  global  features  of  a  nonapproximated  area  segment 
and  a  pointer  to  the  second  list  (symboll  list),  in  which  the  corresponding 
data  for  the  contour  parts  after  contour  approximation  are  contained.  The 
term  contrast  used  in  the  lists  means  the  absolute  value  of  the  gradient  on  the 
contour  and  the  terms  brightness ,  resp.  darkness  refer  to  the  gray  values  of  the 
area  segment. 


5.5  Results  from  Edge  and  Area  Segmentation 

The  segmentation  procedures  introduced  in  this  chapter  are  provided  for  the 
iconic  preprocessing  of  symbolic  blackboard  oriented  structural  image  analysis. 
They  produce  an  enormous  amount  of  edge  and  area  segments  in  each  resolu¬ 
tion  level  of  the  gray  value  pyramid.  Because  of  capacity  it  is  necessary  for  fur¬ 
ther  symbolic  processing  to  reduce  the  large  number  of  segments  by  selecting 
certain  groups  of  segments  as  base  objects  according  to  the  encountered  task. 

For  the  task  of  street  extraction  in  urban  imagery  the  edge  elements  in  the 
best  resolution  level  9  should  be  selected  as  base  elements  for  the  prolonga¬ 
tion  of  short  lines  to  long  lines.  Concerning  areas  the  dark,  resp.  bright  area 
segments  in  the  best  resolution  level  9  projected  from  spot  level  6  should  be 
selected  as  base  elements  for  the  enlargement  of  small  areas  to  large  areas. 


Figure  5.22:  Symbolic  description  of  area  segments 


Chapter  6 

Extended  Production  Model  for 
Street  Network  Extraction 

As  shown  in  chapter  5.  a  huge  amount  of  segmentation  results  is  obtained 
from  improved  image  preprocessing.  Because  of  the  limited  running  time  of  the 
project  the  results  achieved,  however,  could  not  yet  be  used  for  structure  ori¬ 
ented  image  analysis  in  general.  In  order  to  test  the  direct  applicability  of  the 
original  BPI  model  for  line  prolongation  used  in  chapter  4.  (see  Figure  4.9), 
a  section  of  256  x  256  pixels  from  PH0ENIX-2A1  was  segmented  into  edges 
in  the  best  resolution  level  9.  All  edges  received  from  improved  preprocess¬ 
ing  were  equally  assessed  and  stored  in  a  blackboard  as  base  objects  line.  In 
Figure  6.1  those  base  objects  are  plotted  as  thin  lines.  The  base  objects  line 
were  used  to  generate  more  complex  partial  objects  long  line,  plotted  as  thick 
lines  in  Figure  6.1.  The  production  of  long  lines  showed  that  because  of  many 
gaps  between  single  long  lines,  the  target  object  street  network  would  not  be 
extracted  unless  a  modification  of  the  production  P01  is  carried  out.  Further¬ 
more  the  original  model  does  not  allow  to  construct  bending  lines,  so  that  line 
prolongation  and  stripe  composition  would  not  be  done  on  curves  Therefore 
most  of  the  productions  following  P01  also  have  to  be  modified. 

The  direct  application  of  the  original  model  to  edge  elements  received  from 
improved  preprocessing  has  shown  that  the  extraction  of  streets  would  not  be 
successful.  This  is  even  more  true  when  using  edge  and  area  segments  together 
unless  the  model  is  substantially  extended.  Because  of  time  a  general  modifi¬ 
cation  of  the  model  could  not  be  realized.  On  the  other  hand  edge  oriented 
structural  image  analysis  using  the  original  model  has  been  demonstrated  in 
chapter  4.  in  principle.  Therefore  an  extended  model  for  street  network  ex¬ 
traction  was  developed  using  only  the  area  segments  as  base  objects.  Further¬ 
more  the  dark,  resp.  bright  contrasting  small  areas  segmented  by  projecting  the 
spots  from  level  6  to  gray  value  level  9  are  used  only. 

In  the  first  section  of  this  chapter  an  extended  model  for  the  detection  of 
intersections  in  urban  areas  is  described  that  combines  appropriate  small  ar¬ 
eas  to  streets  connected  by  their  intersections.  In  the  next  section  the  set  of 
productions  required  for  connecting  the  partial  objects  is  given.  In  the  last  sec- 
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Figure  6.1:  Analysis  of  a  256  x  256  section  from  PHOENIX.2A1/ 
base  objects:  line  (thin),  partial  objects:  long  line  (thick) 
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Figure  6.2:  Production  net  for  the  area  oriented  detection  of  intersections 

tion  the  results  of  the  street  network  extraction  are  shown  for  the  four  images 
PHOENIX-2A1,  PHOENIX .2B1,  BIETIGHEIM1.31,  and  BIETIGHEIM2.31 
used  in  chapter  4.  for  line  oriented  image  analysis  with  a  BPI  system. 


6.1  Area  Oriented  Detection  of  Intersections 

The  street  network  as  target  object  consists  of  streets  connected  by  their  in¬ 
tersections.  In  the  selected  images  the  streets  appear  as  more  or  less  bending 
homogeneous  stripes.  Those  streets  can  be  constructed,  for  example,  from  big 
and  long  areas  outgoing  from  small  areas  as  base  objects  received  by  the  seg¬ 
mentation  procedure  given  in  chapter  5. 

As  shown  in  Figure  6.2  two  or  more  base  objects  (terminals)  small  area  de¬ 
scribed  by  their  contour  parts  and  radiometric  features  are  connected  to  the 
partial  object  big  area  triggered  by  the  base  object  small  area.  Then  the  partial 
objects  big  area  are  combined  to  the  new  partial  object  long  area,  and  long  ar- 
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eas  build  up  the  partial  objects  street.  If  two  or  more  streets  are  meeting  with 
an  angle  greater  than  a  given  threshold  it  is  assumed  that  there  must  be  an 
object  intersection  to  be  detected.  The  partial  objects  intersection  are  repre¬ 
senting  the  nodes  in  the  target  object  street  network. 

In  the  next  section  the  productions  required  for  generating  the  partial  and 
target  objects  are  given  in  detail. 


6.2  Extended  Set  of  Productions 

In  the  selected  images  the  streets  appear  as  a  network  of  more  or  less  con¬ 
nected  dark  or  bright  long  areas.  This  statement  remains  true  even  for  more 
complex  scenes.  In  the  chosen  resolution  the  width  of  the  long  areas  lies  be 
tween  5  to  12  pixels.  The  long  areas  and  by  this  the  street  network  can  be 
obtained  by  merging  selected  area  segments  received  from  appropriately  chosen 
spot  and  projection  levels.  Before  a  merging  of  segments  can  take  place,  three 
main  criteria  for  selecting  candidates  must  be  fulfilled: 

1.  a  neighborhood  relation  of  the  kind  overlapping  or  adjacent  must  hold  be¬ 
tween  reference  segment  and  candidate  segments 

2.  there  must  be  a  certain  similarity  between  geometric  features  of  reference 
segment  and  candidate  segments 

3.  there  must  also  be  a  certain  similarity  between  radiometric  features  of 
reference  segment  and  candidate  segments 

According  to  the  definitions 

R  -  S,  P)  with 

V’(  =  set  of  base  objects 
V„  =  set  of  partial  objects 
5  =  set  of  target  objects 
P  =  set  of  productions 

given  in  subsection  4.3.1  the  following  productions  are  applied  for  the  detec¬ 
tion  of  intersections  and  extraction  of  the  street  network: 


Vi  =  { small  area } 

Vn  =  {big  area,  long  area,  street,  intersection  } 

S  —  {street  network  } 

P  =  {  PI:  small  area  (small  area)  — >  big  area; 

P 2:  big  area  ( big  area)  — >  long  area; 

P 3:  long  area  ( long  area,  street)  — *  street; 
P 4:  street  (long  area,  street)  — >  street; 


CHAPTER  6.  EXTENDED  PRODUCTION  MODEL 


113 


P 5:  street  (street)  — >  intersection,  ...,  intersection ; 

P 6:  intersection  (street,  intersection)  — »  street  network; 

P 7:  street  network  (street  network)  — >  street  network} 

The  productions  PI  to  P 7  are  described  in  detail  in  the  following  subsec¬ 
tions.  Thereby  the  socalled  triggering  partial  object  denotes  that  object  which 
is  chosen  from  the  supervisor  as  the  (at  present)  best  object  for  furthei  process¬ 
ing  (see  Figure  4.6). 

PI:  Small  Area  — >  Big  Area 

If  a  base  object  small  area  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  big  area  has  to  be  tested.  The  associated  verification  program 
for  testing  that  hypothesis  is  triggered  by  the  base  object  small  area  selected  as 
best  object.  For  testing  purpose  two  examination  steps  are  performed.  First  a 
narrow  search  area  around  that  contour  part  of  the  triggering  small  area  which 
has  the  weakest  contrast  of  all  contour  parts  is  examined.  The  aim  is  to  detect 
contour  parts  of  other  small  areas  which  have  similar  features,  namely  a  sim¬ 
ilar  orientation  and  contrast  as  the  reference  contour  part.  Second,  in  case  of 
success  it  is  tested  whether  the  small  area  belonging  to  the  found  contour  part 
is  overlapping  or  (at  least)  adjacent  with  the  triggering  small  area,  and  having 
similar  mean  brightness  and  mean  contrast.  In  case  of  a  successful  examina¬ 
tion  the  partial  object  big  area  is  built  up  by  the  selected  small  area  together 
with  those  small  areas  found.  The  big  area  is  described  by  the  following  at¬ 
tributes:  endpoints  on  the  contour  of  the  big  area  determined  by  the  regres¬ 
sion  line  laid  through  the  gravity  points  of  all  participating  small  areas,  mean 
brightness,  mean  contrast  to  its  surrounding,  length,  orientation,  mean  width 
and  variance  of  width.  After  composition  of  the  big  area  an  assessment  is  ac¬ 
complished  according  to  its  variance  of  width,  that  means  the  degree  of  fitting 
of  the  small  areas  with  each  other. 

P2:  Big  Area  — *  Long  Area 

If  a  partial  object  big  area  is  selected  as  best  object,  then  the  hypothesis  for  the 
partial  object  long  area  has  to  be  tested.  The  associated  verification  program 
for  testing  that  hypothesis  is  triggered  by  the  partial  object  big  area  selected  as 
best  object.  For  testing  purpose  a  search  area  around  the  big  area  is  examined 
to  find  other  big  areas  with  similar  orientation,  mean  width,  mean  contrast  and 
mean  brightness.  In  case  of  a  successful  examination  the  partial  object  long 
area  is  built  up  by  the  selected  big  area  together  with  those  big  areas  found. 
The  long  area  is  described  by  its  attributes  mean  brightness,  mean  contrast  to 
its  surrounding,  mean  width,  variance  of  width,  orientation  of  the  correspond¬ 
ing  big  areas  at  both  ends,  and  the  length.  After  composition  of  the  long  area 
an  assessment  is  accomplished  according  to  the  variance  of  the  width. 
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P3:  Long  Area  — >  Street 

If  a  partial  object  long  area  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  street  has  to  be  tested.  The  associated  verification  program 
for  testing  that  hypothesis  is  triggered  by  the  partied  object  long  area  selected 
as  best  object.  For  testing  purpose  a  narrow  search  area  around  both  ends  of 
the  long  area  are  examined  to  detect  other  long  areas  or  streets  with  similar 
brightness  and  contrast  values,  similar  width,  and  similar  orientation  at  their 
ends.  In  case  of  a  successful  examination  the  partial  object  street  is  built  up 
by  the  selected  long  area  together  with  those  long  areas  or  streets  found.  The 
street  is  described  by  the  union  of  all  small  areas  belonging  to  that  street.  Af¬ 
ter  composition  of  the  street  an  assessment  is  accomplished  according  to  the 
assessment  of  the  triggering  long  area. 

P4:  Street  — *  Street 

If  a  partial  object  street  is  selected  as  best  object,  then  the  hypothesis  for  the 
partial  object  street  has  to  be  tested  in  addition  to  the  hypothesis  P 5.  The 
associated  verification  program  for  testing  that  hypothesis  is  triggered  by  the 
partial  object  street  selected  as  best  object.  For  testing  purpose  a  narrow 
search  area  around  the  street  is  examined  to  detect  other  long  areas  or  streets 
with  similar  brightness  and  contrast  values,  similar  width,  and  similar  orienta¬ 
tion.  In  case  of  a  successful  examination  the  par*ial  object  street  is  built  up 
by  the  selected  street  together  with  those  long  araes  and  r  .s  f»uu  1.  After 
composition  of  the  street  an  assessment  is  accomo’i'iieu  according  to  the  as¬ 
sessment  of  the  triggering  street. 

P5:  Street  — ♦  Intersection,  I  it.ersect.ion 

If  a  partial  object  street  is  selected  as  best  object,  then  the  hypothesis  for 
the  partial  object  intersection  has  to  be  tested  in  addition  to  the  hypothesis 
P4.  The  associated  verification  program  for  testing  that  hypothesis  is  triggered 
by  the  partial  object  street  selected  as  best  object.  For  testing  purpose  many 
search  areas  originating  from  the  positions  of  the  long  areas  along  the  selected 
street  are  oriented  at  different  angles  with  respect  to  the  direction  of  the  street. 
The  selected  street  is  used  to  find  other  streets  which  have  a  crossing  with  this 
street  within  the  image.  The  streets  found  have  to  meet  the  criterion  closeness 
to  the  selected  street.  In  case  of  a  successful  examination  the  partial  objects  in¬ 
tersection  are  built  up  by  the  selected  street  together  with  those  streets  found. 
After  composition  of  the  intersection  an  assessment  is  accomplished  according 
to  the  above  mentioned  criterion. 

P0:  Intersection  — ♦  Street  Network 

If  a  partial  object  intersection  is  selected  as  best  object,  then  the  hypothesis 
for  the  target  object  street  network  has  to  be  tested.  The  associated  verification 
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program  for  testing  that  hypothesis  is  triggered  by  the  partial  object  intersec¬ 
tion  selected  as  best  object.  For  testing  purpose  all  streets  belonging  to  the 
selected  intersection  are  accessed.  The  target  object  street  network  is  built  up 
by  the  selected  intersection  together  with  thoie  streets  accessed.  After  compo¬ 
sition  of  the  street  network  no  particular  assessment  is  accomplished. 

P7:  Street  Network  — >  Street  Network 

All  partied  objects  intersection  composed  by  testing  the  hypothesis  P 6  are  as¬ 
sembled.  The  intersections  with  their  associated  streets  can  be  accessed  by 
searching  for  assessment  values  greater  than  the  threshold  given  for  successful 
examination  of  intersections. 


6.3  Results  from  Area  Oriented  Extraction  of 
Street  Networks 

In  order  to  test  the  extended  model  for  the  street  network  analysis  described  in 
the  previous  section  the  same  images  as  in  chapter  4.  were  used: 

•  PH0ENIX-2A1  (1/4  section  of  PHOENIX-2,  see  Figure  4.10) 

512  x  512  pixels 
1  :  20,000  real  scale 
50  /tm  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  PHOENIX.2B1  (1/4  section  of  PHOENIX-2,  see  Figure  4.14) 

512  x  512  pixels 
1  :  20,000  real  scale 
50  fim  pixel  size  (on  image) 

1  m  pixel  size  (on  earth) 

•  BIETIGHEIM1.31  (1/4  section  of  BIETIGHEIM1.3,  sec  Figure  4.18) 

512  x  512  pixels 
1  :  14,000  real  scale 
50ptm  pixel  size  (on  image) 

0.7m  pixel  size  (on  earth) 

•  BIETIGHEIM2.31  (1/4  section  of  BIETIGHEIM2-3,  see  Figure  4.22) 


512  x  512  pixels 
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1  :  4,000  real  scale 

100  fim  pixel  size  (on  image) 

0.4  m  pixel  size  (on  earth) 

The  results  from  all  test  images  are  presented  in  the  same  way  (see  Figure  6.3 
to  Figure  6.30): 

1.  preprocessing  of  test  image/base  objects:  small  area  (blue) 

2.  analysis  of  test  image/generated  partial  objects:  big  area  (yellow)  trig¬ 
gered  by  small  area 

3.  analysis  of  test  image/generated  partial  objects:  long  area  (green)  trig¬ 
gered  by  big  area 

4.  analysis  cf  test  image/generated  partial  objects:  street  (green)  triggered 
by  long  area  or  street 

5.  analysis  of  test  image/generated  partial  objects:  intersection  (red)  trig¬ 
gered  by  street 

6.  analysis  of  test  image/generated  target  objects:  street  network  (yellow) 
triggered  by  intersection  or  street  network 

7.  analysis  of  test  image/target  objects:  street  network  (yellow),  partial  ob¬ 
jects:  intersection  (red),  noncomposing  partial  objects:  street  (green), 
long  area  (green),  big  area  (yellow) 

Every  last  figure  shows  the  complete  extraction  result  of  the  street  network 
analysis  where  the  generated  partial  objects  big  area,  long  area,  street,  intersec¬ 
tion,  and  the  target  object  street  network  are  superimposed  such  that  only  the 
noncomposing  partial  objects  are  seen  and  the  underlying  composing  objects 
are  not  seen. 
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6.3.1  PHOENIX  _2A1 

Because  of  its  quite  clear  structures  this  scene  was  used  to  develop  and  test 
the  production  modules  and  to  adjust  the  parameters  for  the  structure  oriented 
image  analysis  with  a  blackboard  system.  In  this  scene  the  street  network  con¬ 
sists  of  more  or  less  dark  contrasting  long  areas.  Therefore  the  dark  areas  seg¬ 
mented  by  projecting  the  spots  from  level  6  to  gray  value  level  9  are  used  from 
preprocessing  and  applied  as  base  objects  small  area  for  structural  analysis. 

As  shown  in  Figure  6.3,  bigger  areas  in  the  street  network  (e.  g.  intersec¬ 
tions  of  streets)  sometimes  are  not  segmented  from  projection  level  9  with  a 
level  difference  of  3.  The  generated  partial  objects  big  area  are  shown  in  Fig¬ 
ure  6.4.  The  isolated  small  areas  which  could  not  be  connected  to  big  areas  are 
thrown  away.  Figure  6.5  shows  all  partial  objects  long  area.  Those  big  areas 
which  do  not  meet  the  connection  rules  are  suppressed. 

In  Figure  6.6  the  partial  objects  street  generated  by  connecting  long  areas 
or  streets  are  shown.  Only  few  areas  in  the  fields  and  some  segments  of  roofs 
are  detected  as  streets.  On  the  other  hand  some  parts  of  the  streets  are  not 
found  because  of  their  inhomogeneities  with  respect  to  gray  values  or  shape 
and  because  of  misadaption  to  the  width  of  the  street  (level  difference  of  3). 

Figure  6.7  shows  the  position  of  the  detected  objects  intersection.  Ten  in¬ 
tersections  are  found.  All  streets  which  are  connected  by  these  intersections  are 
plotted  in  the  next  figure.  Figure  6.8  represents  the  network  of  streets  origi¬ 
nating  from  the  intersections.  The  street  network  is  detected  nearly  completely 
with  respect  to  the  adapted  width  of  streets.  In  Figure  6.9  all  object  types 
from  big  area  up  to  the  street  network  are  superimposed.  Only  the  base  objects 
( small  area)  are  omitted  for  clearness. 


Figure  6.3:  Preprocessing  of  PH0ENIX-2Al/base  objects:  small  area  (blue) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50/xm  :  1m) 
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Figure  6.4:  Analysis  of  PH0ENIX-2Al/triggering  base  objects:  small  area, 
generated  partial  objects:  big  area  (yellow) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50pm  :  lm) 
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Figure  6.5:  Analysis  of  PHOENlX_2Al/triggering  partial  objects:  big  area, 
generated  partial  objects:  long  area  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 /jm  :  lm) 
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Figure  6.6:  Analysis  of  PH0ENIX-2A1  /triggering  partial  objects:  long  area, 
street,  generated  partial  objects:  street  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50/jm  :  1  m) 
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Figure  6.7:  Analysis  of  PH0ENIX-2Al/triggering  partial  objects:  street, 
generated  partial  objects:  intersection  (red) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fsm  :  1  m) 
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Figure  6.8:  Analysis  of  PH0ENIX-2Al/triggering  partial  objects:  intersection, 
street  network,  generated  partial  objects:  street  network  (yellow) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  1m) 


—  Jx 
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6.3.2  PHOENIX -2B1 

As  in  PHOENIX.2A1  the  dark  contrasting  small  areas  segmented  by  project¬ 
ing  the  spots  from  level  6  to  gray  value  level  9  are  taken  as  base  objects.  The 
adjustment  of  the  parameters,  however,  is  modified  with  respect  to  a  smaller 
variance  of  the  width  of  long  areas  for  the  analysis  of  PH0ENIX-B1.  The 
street  network  is  detected  nearly  completely  with  respect  to  the  adapted  width 
of  streets  as  in  PH0ENIX.2A1  (see  Figure  6.10  to  Figure  6.16).  Some  con¬ 
necting  dark  areas  are  interpreted  as  streets,  thus  misdetection  of  intersections 
could  not  be  avoided  in  a  few  cases. 

6.3.3  BIETIGHEIM131 

In  this  scene  the  bright  contrasting  small  areas  segmented  by  projecting  the 
spots  from  level  6  to  gray  value  level  9  are  taken  as  base  objects.  In  order 
to  test  the  applicability  of  the  BPI  model  and  the  adjustment  of  the  parame¬ 
ters  to  this  more  complex  scene,  the  same  parameter  setting  as  for  the  analysis 
of  the  scene  PHOENIX.2B1  is  taken.  Because  of  misadjustment  with  respect 
to  the  width  of  streets  one  can  expect  that  the  results  are  worse  than  from 
PH0ENIX_2A1  and  PH0ENIX_2B1.  Though  the  result  of  the  street  network 
extraction  seems  quite  poor,  no  misdetection  of  intersections  occurs  (see  Fig¬ 
ure  6.17  to  Figure  6.23). 

Figure  6.23  shows  that  especially  the  broader  streets,  where  the  base  ob¬ 
jects  small  area  (Figure  6.17)  are  lying  side  by  side  and  not  only  one  behind 
the  other,  are  not  extracted  very  well.  This  is  due  to  the  fact  that  the  con¬ 
nection  from  big  areas  (Figure  6.18)  to  long  areas  (Figure  6.19)  and  then  up  to 
streets  (Figure  6.20)  fails. 

6.3.4  BIETIGHEIM231 

As  in  BIETIGHEIM1.31  the  bright  contrasting  small  areas  segmented  by  pro¬ 
jecting  the  spots  from  level  6  to  gray  value  level  9  are  taken  as  base  objects. 
Because  of  misadjustment  with  respect  to  the  width  of  streets  one  can  expect 
that  the  results  as  in  BIETIGHEIM1-31  are  worse  than  from  PHOENIX-2A1 
and  PH0ENIX_2B1.  Though  the  BPI  model  and  the  parameter  setting  are  not 
changed,  the  result  of  the  analysis  from  BIETIGHEIM2_31  is  obvious  better 
than  that  of  the  scene  BIETIGHEIMl-31.  One  misdetection  of  an  intersection 
occurs  in  the  horizontal  street  in  the  lower  part  of  the  figure.  Some  streets  are 
not  taken  in  their  total  width.  This  is  due  to  the  strong  variation  of  their  gray 
values. 
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Figure  6.10:  Preprocessing  of  PHOENIX_2Bl/base  objects:  small  area  (blue) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50pm  :  lm) 
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Figure  6.13:  Analysis  of  PH0ENIX_2Bl/triggering  partial  objects:  long  area, 
street,  generated  partial  objects:  street  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  bOfim  :  1  m) 


F  i 


Figure  6.14:  Analysis  of  PH0ENIX.2Bl/triggering  partial  objects:  street, 
generated  partial  objects:  intersection  (red) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  5Q(im  :  lm) 


Figure  6,15:  Analysis  of  PH0ENIX-2Bl/triggering  partial  objects: 
intersection,  street  network,  generated  partial  objects:  street  network  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  1  m) 
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Figure  6.16:  Analysis  of  PH0ENIX_2Bl/target  objects:  ateet  network  (yellow), 

partial  objects:  intersection  (red),  street  (green), 

noncomposing  partial  objects:  long  area  (green),  big  area  (yellow) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fam  :  lm) 
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Figure  6.17:  Preprocessing  of  BIETIGHEIMl.31/base  objects:  small  area 
(blue)  (matrix  size  512  x  512  pixels,  pixel  size  and  scale  50/um  :  0.7m) 


Figure  6.20:  Analysis  of  BIETIGHEIM1.31/triggering  partial  objects: 
long  area,  street,  generated  partial  objects:  street  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 fim  :  0.7 m) 


Figure  6.21:  Analysis  of  BIETIGHEIM1  -31 /triggering  partial  objects:  street, 
generated  partial  objects:  intersection  (red) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 pm  :  0.7m) 
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Figure  6.22:  Analysis  of  BIETIGHEIM1.31/triggering  partial  objects: 
intersection,  street  network,  generated  partial  objects:  street  network  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 (im  :  0.7 m) 
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Figure  6.23:  Analysis  of  BIETIGHEIMl_31/target  objects: 
steet  network  (yellow),  partial  objects:  intersection  (red),  street  (green), 
noncomposing  partial  objects:  long  area  (green),  big  area  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  50 /jm  :  0.7m) 
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Figure  6.24:  Preprocessing  of  BIETIGHEIM2-31/base  objects:  small  area 
(blue)  (matrix  size  512  x  512  pixels,  pixel  size  and  scale  lOO^ro  :  0.4 m) 
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Figure  6.26:  Analysis  of  BIETIGHEIM2-31/triggering  partial  objects:  big  area , 
generated  partial  objects:  long  area  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  lOOfim  :  0.4m ) 


Figure  6.27:  Analysis  of  BIETIGHEIM2-31  /triggering  partial  objects: 
long  area,  street,  generated  partial  objects:  street  (green) 

(matrix  size  512  x  512  pixels,  pixel  size  and  scale  lOOpm  :  0.4m) 
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Figure  6.29:  Analysis  of  BIETIGHEIM2_31/triggering  partial  objects: 
intersection,  street  network,  generated  partial  objects:  street  network  (yellow) 
(matrix  size  512  x  512  pixels,  pixel  size  and  scale  lOO^rn  :  0.4m) 


Chapter  7 
Conclusions 


The  results  about  street  network  extraction  in  urban  imagery  achieved  so  far 
from  solutions  of  low,  medium,  and  high  level  image  processing  are  discussed 
and  some  concluding  remarks  are  presented  in  the  following: 

•  The  line  extraction  method  using  stream  following  works  quite  well  when 
applied  to  image  scales  where  the  line  objects  really  appear  as  line 
shaped  objects  (only  a  few  pixels  wide).  In  urban  areas  the  detection 
of  those  objects  is  more  difficult  than  in  rural  scenes  because  of  greater 
noise  from  variations  of  shape  and  gray  value  (contrast).  Therefore  an 
image  has  to  be  processed  with  different  sets  of  parameters  and  with 
changing  image  resolution  in  order  to  find  most  of  the  line  objects.  With 
respect  to  the  extraction  of  street  networks  misrecognition  of  streets  and 
their  intersections  may  occur  because  no  image  understanding  knowledge 
is  applied. 

•  The  line  extraction  method  using  structured  parallel  image  operations 
(SIO)  produces  similar  results  as  compared  with  that  using  stream  fol¬ 
lowing.  However,  the  SIO  method  is  more  robust  with  respect  to  param¬ 
eter  adjustment,  dearer  to  parameter  adaption,  and  more  flexible  to  the 
extension  of  models  and  algorithms.  Although  no  essential  difference  ex¬ 
ists  taking  the  sequential  or  parallel  image  processing  method,  the  paral¬ 
lel  approach  should  be  favored  because  of  its  easier  application. 

•  The  edge  oriented  segmentation  method  produces  more  complete  results 
than  the  preprocessing  procedure  for  line  primitives  of  given  length.  The 
completeness  is  meant  in  the  sense  of  a  higher  degree  of  the  multiresolu- 
tional  radiometric  and  geometric  description  of  an  edge  segment. 

•  The  area  oriented  segmentation  method  produces  also  more  complete  re¬ 
sults  than  the  preprocessing  procedure  for  line  primitives  of  given  length. 
The  completeness  is  meant  in  the  sense  of  a  higher  degree  of  the  multires- 
olutiona!  radiometric  and  geometric  description  of  an  area  segment.  Fur¬ 
thermore  the  area  oriented  segmentation  method  supplements  the  edge 
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oriented  segmentation  method.  If  only  one  method  is  to  be  applied  then 
the  edge  oriented  segmentation  should  be  preferred  because  of  more  clear¬ 
ness  of  its  elements  compared  to  those  from  area  oriented  segmentation. 

•  In  order  to  extract  street  networks  in  urban  areas  with  higher  perfor¬ 
mance  more  intelligent,  knowledge  based  image  understanding  methods 
are  used.  The  complexity  of  the  first  models  used  for  blackboard  oriented 
structural  image  analysis  (BPI)  is  still  low  and  depends  on  the  applied 
preprocessing  method.  Depending  on  the  kind  of  line  primitives  from  low 
level  and  of  base  objects  (edge  or  area  segments)  from  medium  level  pro¬ 
cessing,  different  BPI  models  and  sets  of  productions  are  (partly)  devel¬ 
oped  and  tested.  Because  of  running  out  of  time  it  was  decided  on  the 
one  hand  to  use  line  primitives  and  on  the  other  hand  only  one  type  of 
base  objects  (area  segments)  in  only  one  projection  level  for  structural 
analysis. 

•  In  case  of  preprocessed  images  providing  the  BPI  system  with  line  prim¬ 
itives  no  BPI  model  was  developed  but  an  existing  model  adapted  only 
to  the  task  of  street  network  recogniLion.  Therefore  no  curved  streets  are 
extracted  because  they  are  not  provided  by  the  model.  Inspite  of  that 
the  results  obtained  from  the  scenes  with  straight  streets  and  their  inter¬ 
sections  show  already  the  quality  of  advanced  recognition  inherent  in  the 
BPI  method. 

•  In  case  of  preprocessed  images  providing  the  BPI  system  with  area  seg¬ 
ments  an  existing  BPI  model  was  modified  adapting  and  interfacing  the 
lower  levels  of  the  model  to  the  preprocessing.  This  model  provides 
curved  street  as  well.  However,  the  complexity  inherent  in  the  BPI  model 
and  in  the  provided  descriptions  of  the  area  segments  are  not  vet  used 
appropriately  for  street  network  recognition.  Even  the  preliminary  results 
show  that  in  principle  the  quality  of  the  structural  image  analysis  is  much 
higher  than  that  which  can  be  received  from  the  line  extraction  by  low 
level  processing. 

The  test  imagery  has  two  levels  of  complexity.  The  urban  structures  of  Bie- 
tigheim  are  more  complicated  than  those  of  Phoenix.  Comparisons  between 
low  level  line  extraction  and  high  level  street  network  analysis  show,  that  an  al¬ 
ready  simple  image  processing  method  produces  rather  good  results  from  both 
types  of  scenes.  It  could  be  shown  that  application  of  the  developed  and  imple¬ 
mented  improved  methods  and  models  produce  better  results  in  case  of  radio- 
metric  and  geometric  ambiguities  of  image  structures.  However,  the  more  ex¬ 
pensive  the  preprocessing  methods  are,  the  more  complex  the  high  level  models 
have  to  be  in  order  to  make  use  of  all  relevant  base  objects  produced  by  low 
level  processing.  Furthermore  the  dependence  of  the  results  on  the  type  of  a 
scene  is  greater  in  this  case  than  in  the  case  of  low  level  line  extraction. 
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At  the  moment  no  attention  was  paid  to  the  problems  of  optimization  of 
computer  run  time  and  capacity  of  memory.  In  this  respect,  optimization  has 
to  be  done  with  regard  to  the  refinement  of  the  software  as  well  as  to  the  ap¬ 
plication  of  special  blackboard  techniques  realized  in  hardware.  For  further 
investigations  in  street  network  extraction  from  aerial  imagery  a  combination 
is  favored  of  the  preprocessing  by  SIO  operations  usin^  the  procedures  of  im¬ 
proved  segmentation  with  the  BPI  analysis  using  a  more  advanced  structural 
BPI  model. 
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